Top RAG Papers

RuleRAG - This paper presents Rule-Guided RAG which explicitly introduces symbolic rules as demonstrations for in-context learning to guide retrievers to retrieve logically related documents. Experiments demonstrate that training-free RuleRAG-ICL effectively improves the retrieval quality and generation accuracy over standard RAG. Moreover, RuleRAG scales well with increasing numbers of retrieved documents and exhibits generalization ability for untrained rules. Read more
ChunkRAG - This paper introduces ChunkRAG, a novel chunk filtering method for RAG systems. Existing RAG methods fail to effectively filter irrelevant chunks which can result in inaccurate responses. ChunkRAG employs semantic chunking to divide documents into coherent sections and utilizes LLM-based relevance scoring to assess each chunk’s alignment with the user’s query. Experiments demonstrated that ChunkRAG outperforms existing RAG models. Read more
AutoRAG Framework - This paper introduces the AutoRAG framework, which automatically identifies suitable RAG modules for a given dataset. AutoRAG explores and approximates the optimal combination of RAG modules for the dataset. AutoRAG is available as a Python library. Read more
LONG²RAG - Existing RAG benchmarks struggle to evaluate long-context retrieval and lack a comprehensive evaluation method for assessing LLMs’ ability to generate long-form responses. To address these shortcomings, this paper introduces the LONG2RAG benchmark and the Key Point Recall (KPR) metric. KPR evaluates the extent to which LLMs incorporate key points extracted from the retrieved documents into their generated responses. Read more
EasyRAG - This paper presents EasyRAG, a simple, lightweight, and efficient retrieval-augmented generation framework for automated network operations. EasyRAG has three advantages, namely, accurate question answering, simple deployment, and efficient inference. Read more
Vision-based RAG (VisRAG) - This paper presents VisRAG which involves embedding the document using a VLM as an image and then retrieving it to enhance the generation of a VLM. When compared to traditional text-based RAG, VisRAG maximizes the retention and utilization of the data information in the original documents, eliminating the information loss introduced during the parsing process. VisRAG outperforms traditional RAG in both the retrieval and generation stages, achieving a 25–39% end-to-end performance gain over the traditional text-based RAG pipeline. Read more
FunnelRAG - This paper presents a progressive retrieval paradigm with coarse-to-fine granularity for RAG, termed FunnelRAG. FunnelRAG achieves comparable retrieval performance while the time overhead is reduced by nearly 40 percent. Read more
CoFE-RAG - This paper presents CoFE-RAG, a Comprehensive Full-chain Evaluation framework to facilitate thorough evaluation across the entire RAG pipeline, including chunking, retrieval, reranking, and generation. CoFE-RAG provides unique insights into the effectiveness of RAG systems in handling diverse data scenarios. Read more
RAG-based Spelling Correction - This paper presents a RAG-based spelling correction approach for E-Commerce applications. In this approach, product names are retrieved from a catalog and incorporated into the context used by a large language model (LLM) that has been fine-tuned to do contextual spelling correction. Results show improvements in spelling correction utilizing the RAG framework beyond a stand-alone LLM. Read more
Multilingual RAG Benchmark (MIRAGE-Bench) - This paper introduces MIRAGE-Bench, a standardized arena-based multilingual RAG benchmark for 18 diverse languages on Wikipedia. The benchmark is constructed using MIRACL, a retrieval dataset, and extended for multilingual generation evaluation. Results show that proprietary and large open-source LLMs currently dominate in multilingual RAG. Read more
Assessing RAG Models for Health Chatbots - This paper presents an extensive assessment of 24 LLMs on real-world data collected from Indian patients interacting with a medical chatbot in Indian English and four other Indic languages. Results show that models vary significantly in their performance, instruction-tuned Indic models do not always perform well on Indic language queries, and factual correctness is generally lower for responses to Indic queries compared to English queries. Read more
Medical Multimodal RAG System - This paper presents MMed-RAG, a versatile multimodal RAG system, designed to enhance the factuality of Med-LVLMs. MMed-RAG can achieve an average improvement of 43.8% in the factual accuracy of Med-LVLMs. Read more
Enhancing E-commerce Product Title Translation with RAG - This paper introduces a RAG-based approach to enhance LLM-based product title translation. The approach leverages existing bilingual product information in e-commerce by retrieving similar bilingual examples. Experiment results show the proposed approach improves product title translation quality by up to 15.3%. Read more
Enhancing LLM-based ASR Accuracy with RAG - This paper presents a RAG-based approach called LA-RAG to enhance automatic speech recognition (ASR) accuracy. LA-RAG leverages fine-grained token-level speech datastores and a speech-to-speech retrieval mechanism to enhance ASR accuracy. Experiments on Mandarin and various Chinese dialect datasets demonstrate significant improvements in ASR accuracy. Read more
SFR-RAG - This paper introduces SFR-RAG, a small LLM that is instruction-tuned with an emphasis on context-grounded generation and hallucination minimization. SFR-RAG-9B model outperforms Command-R+ (104B) and GPT-4o, achieving state-of-the-art results in 3 out of 7 benchmarks. Read more
RAG for AI Legal and Policy Applications - This paper introduces a Hybrid Parameter-Adaptive RAG (HyPA-RAG) system tailored for AI legal and policy. HyPA-RAG uses a query complexity classifier for adaptive parameter tuning, a hybrid retrieval strategy combining dense, sparse, and knowledge graph methods, and an evaluation framework with specific question types and metrics. Experiment results show enhanced correctness, faithfulness, and contextual precision. Read more
Impact of Fair Ranking in RAG - Retrieval is a core component of RAG systems. This paper studies the impact of fair ranking in RAG systems by evaluating nine different RAG systems that incorporate fair rankings across seven distinct datasets. Experiment results show that RAG systems with fair rankings can maintain a high level of generation quality and even outperform traditional RAG systems in many cases. Read more
Trustworthiness in RAG (Survey) - RAG enhances LLMs by providing them with useful and up-to-date knowledge from vast external databases. However, RAG systems are at the risk of generating undesirable contents if the retrieved information is either inappropriate or poorly utilized. This paper presents a comprehensive survey of trustworthiness in RAG systems identifying gaps and highlighting promising approaches. Read more
Meta Knowledge for RAG - The authors introduce a data-centric RAG workflow, which involves generating metadata and synthetic QA for documents. Results demonstrate that meta knowledge-augmented queries improve retrieval precision, recall, and answer quality. The methodology is cost-effective, requiring less than $20 per 2000 research papers using Claude 3 Haiku. Read more
RAGLab - RAGLab is a modular, research-oriented open-source library that includes the implementation of six existing RAG algorithms. It provides a comprehensive ecosystem for investigating RAG algorithms, addressing the constraints in RAG development. Read more
GraphRAG (Survey) - Retrieval-Augmented Generation (RAG) addresses LLM challenges but struggles to handle complex entity relationships in databases. GraphRAG addresses this by leveraging structural information for more precise retrieval and context-aware responses. This paper presents the first comprehensive overview of GraphRAG methodologies and also explores applications, evaluation methods, and future research directions. Read more
CommunityKG-RAG - This paper introduces CommunityKG-RAG which integrates community structures within Knowledge Graphs with Retrieval-Augmented Generation systems to enhance fact-checking. CommunityKG-RAG can adapt to new domains and queries without additional training, making it highly versatile and applicable across various contexts. Read more
W-RAG - Training of dense retrieval in RAG systems is challenging due to the scarcity of ground-truth evidence. This paper introduces W-RAG, which utilizes LLMs' ranking capabilities to create weakly labeled data for training dense retrievers. W-RAG enhances both retrieval and OpenQA performance compared to baseline models. Read more
Inference Scaling for Long-Context Retrieval Augmented Generation - The paper "Inference Scaling for Long-Context Retrieval Augmented Generation" addresses challenges in scaling RAG models for long-context tasks by proposing techniques like in-context learning and iterative prompting. These methods help RAG models efficiently manage and retrieve information across extensive document contexts. The authors introduce an inference scaling model that dynamically optimizes compute allocation, achieving nearly linear scaling. Key techniques include context chunking, adaptive retrieval, and computation-efficient inference, resulting in a 58.9% improvement over standard approaches on long-context benchmarks. Read more
Astute RAG - This paper introduces Astute RAG which improves robustness in Retrieval-Augmented Generation systems by addressing imperfect retrieval and knowledge conflicts. Using source-awareness, reliability assessment, and adaptive knowledge consolidation, Astute RAG enhances response accuracy through techniques like confidence-weighted retrieval, adaptive filtering, and cross-source validation. Read more
Long-Context LLMs Meet RAG - This paper addresses the challenges of processing long-context inputs in Retrieval-Augmented Generation (RAG) systems. It explores techniques like retrieval of hard negatives and proposes training-free methods, such as retrieval reordering, alongside training-based methods like implicit LLM fine-tuning and intermediate reasoning tasks. The study also investigates the impact of data distribution and retriever selection for optimal performance in long-context scenarios. Read more
Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make Your LLMs Use External Data More Wisely - This survey explores techniques for integrating external data with large language models to improve accuracy, reduce hallucinations, and enhance domain-specific performance. It categorizes data needs from simple retrieval to complex reasoning and suggests approaches for deploying RAG in specialized domains. Read more
- Data Categorization - Organizes user queries based on data needs, ranging from simple fact retrieval to complex reasoning.
- Memory Management - Optimizes external data storage and access to improve retrieval efficiency.
- Data Filtering - Filters noisy or irrelevant data to maintain high-quality retrieval.
- Contextual Re-ranking - Prioritizes retrieved information by aligning it closely with the query context.
- Hybrid Retrieval - Integrates structured data (like knowledge graphs) with unstructured sources for more robust responses.
- Knowledge Base Integration - Incorporates structured knowledge to enhance model reliability and reduce hallucinations.
In Defense of RAG in the Era of Long-Context Language Models - This paper introduces Order-Preserve RAG (OP-RAG), a retrieval-augmented generation method that preserves the original order of retrieved text chunks. By maintaining the sequence, OP-RAG improves answer quality, addressing issues seen in long-context models that degrade with irrelevant information. OP-RAG strikes a balance between relevance and precision, outperforming traditional approaches in quality. Read more
Pandora’s Box or Aladdin’s Lamp: A Comprehensive Analysis Revealing the Role of RAG Noise in Large Language Models - This paper introduces NoiserBench, a benchmark to categorize noise types in Retrieval-Augmented Generation (RAG) systems as beneficial or harmful. Beneficial noise, like semantic or datatype noise, can improve model capabilities, while harmful noise, such as counterfactual or orthographic noise, impairs performance. These insights aid in designing more robust RAG systems. Read more
Agentic Retrieval-Augmented Generation for Time Series Analysis - This paper introduces an Agentic RAG framework for time series analysis, employing a hierarchical multi-agent system with a master agent and specialized sub-agents for task delegation. Using prompt pools and historical patterns, it enhances prediction accuracy for complex, shifting data. Techniques like Direct Preference Optimization and ReAct prompting are applied to improve forecasting and anomaly detection outcomes. Read more
GraphRAG: Enhancing Retrieval-Augmented Generation with Graph Structure - This paper introduces GraphRAG, which integrates graph-based structures into RAG models to leverage complex entity relationships for improved retrieval and context-aware responses. Key components include **entity linkage**, **graph-based embeddings**, and **knowledge graph integration**, which collectively enhance data retrieval accuracy. The paper provides an overview of GraphRAG methodologies, application areas, and evaluation techniques for further research. Read more
EfficientRAG: Efficient Retriever for Multi-Hop Question Answering - This paper introduces EfficientRAG, a retrieval system tailored for multi-hop question answering. Using a Labeler and Filter mechanism, EfficientRAG generates new queries iteratively, filtering relevant information to improve retrieval efficiency and reduce computational costs. Experimental results demonstrate that EfficientRAG outperforms existing multi-hop QA systems by boosting precision and accuracy on open-domain datasets. Read more
RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation - This paper introduces RAGChecker, a framework designed for fine-grained evaluation and diagnosis of Retrieval-Augmented Generation (RAG) systems. RAGChecker provides a suite of diagnostic metrics for both retrieval and generation components, allowing for detailed performance analysis. Meta-evaluation results indicate that RAGChecker correlates more strongly with human judgments than existing metrics. The authors use RAGChecker to assess eight RAG systems, uncovering valuable insights into design choices and trade-offs in RAG architectures. Read more
HybridRAG: Integrating Knowledge Graphs and Vector Retrieval Augmented Generation for Efficient Information Extraction - This paper introduces HybridRAG, a model that combines VectorRAG and GraphRAG to enhance retrieval and accuracy in information extraction tasks, especially for complex financial documents. By integrating knowledge graphs with vector-based retrieval, HybridRAG improves context and relevance, outperforming standalone models in financial question answering. The model demonstrates potential applications across various domain-specific tasks. Read more
Medical Graph RAG: Towards Safe Medical Large Language Model via Graph Retrieval-Augmented Generation - This paper introduces MedGraphRAG, a retrieval-augmented generation framework tailored for medical applications, utilizing a three-tier hierarchical graph structure for organizing medical data. Key features include semantic document segmentation, entity extraction, and U-retrieve to ensure balanced retrieval. MedGraphRAG improves accuracy on medical Q&A benchmarks, delivering source-cited, evidence-based responses that enhance clinical safety and transparency. Read more
RAG Foundry: A Framework for Enhancing LLMs for Retrieval Augmented Generation - This paper introduces RAG Foundry, an open-source framework designed to streamline the development and evaluation of retrieval-augmented generation models. It integrates data augmentation, prompt design, fine-tuning, and chain-of-thought reasoning into a unified workflow, supporting rapid prototyping and experimentation. RAG Foundry enhances knowledge-intensive tasks by offering a modular approach to various RAG configurations. Read more
RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework - This paper introduces RAGEval, a framework designed to assess Retrieval-Augmented Generation (RAG) systems across diverse scenarios by generating high-quality documents, questions, answers, and references through a schema-based pipeline. With a focus on factual accuracy, it proposes three novel metrics—Completeness, Hallucination, and Irrelevance—to rigorously evaluate LLM-generated responses. Experimental results show that RAGEval outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples. Furthermore, the use of LLMs for scoring the proposed metrics demonstrates a high level of consistency with human evaluations. RAGEval establishes a new paradigm for evaluating RAG systems in real-world applications. Read more
Improving Retrieval Augmented Language Model with Self-Reasoning - This paper presents a self-reasoning framework to enhance the reliability and traceability of Retrieval-Augmented Language Models (RALMs). The framework includes relevance-aware, evidence-aware, and trajectory analysis processes that guide RALMs in selecting relevant documents and generating reasoning pathways. This approach improves accuracy in question-answering and fact-verification tasks, achieving strong results with minimal training data. Read more
Adaptive Retrieval-Augmented Generation for Conversational Systems - This paper introduces RAGate, a gating model that selectively decides when to use external knowledge in responses, optimizing the quality of conversational systems. By modeling context and utilizing human judgments, RAGate minimizes unnecessary augmentation, reducing irrelevant or repetitive information. Tests on conversational datasets demonstrate that RAGate enhances response accuracy and relevance, improving the user experience in task-oriented dialogues. Read more
Context Embeddings for Efficient Answer Generation in RAG - This paper introduces COCOM, a context compression model designed to improve the efficiency of Retrieval-Augmented Generation (RAG) by reducing lengthy contexts into compressed embeddings. COCOM supports adjustable compression rates, balancing answer quality and generation speed, and achieves up to 5.69x faster generation times. Key features include multi-context compression and efficient answer generation through adaptive pretraining, outperforming existing methods in speed and accuracy. Read more
Exploring Advanced Large Language Models with LLMSuite - This paper offers a comprehensive overview of methods and architectures for enhancing large language models (LLMs). It covers Retrieval Augmented Generation (RAG), Program-Aided Language Models (PAL), and tools like ReAct and LangChain to address knowledge cutoffs and accuracy issues. The paper also explores fine-tuning strategies, including LoRA, RLHF, and ReST, for improved multi-step reasoning and complex task performance. Read more
RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs - This paper introduces RankRAG, an instruction fine-tuning framework that allows a single large language model (LLM) to perform both context ranking and answer generation within retrieval-augmented generation (RAG) systems. By training on a small fraction of ranking data, RankRAG surpasses existing expert ranking models and demonstrates strong performance across nine knowledge-intensive benchmarks, performing comparably to GPT-4 on five biomedical RAG benchmarks without needing domain-specific fine-tuning. Read more
Searching for Best Practices in Retrieval-Augmented Generation - This paper investigates existing RAG approaches and their potential combinations to identify optimal practices for deploying retrieval-augmented generation (RAG) systems. The study examines how RAG techniques effectively integrate up-to-date information, mitigate hallucinations, and improve response quality in specialized domains. It suggests strategies to balance performance and efficiency, and highlights the benefits of multimodal retrieval techniques for enhancing question-answering and generating multimodal content using a "retrieval as generation" approach. Read more
Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems - This paper argues for the use of summarization as a robust evaluation method for long-context language models (LLMs) and retrieval-augmented generation (RAG) systems. It introduces the "Summary of a Haystack" (SummHay) task, requiring models to process a synthesized collection of documents (a haystack) and generate a summary that captures relevant insights and cites source documents. The authors develop a reproducible evaluation framework that measures performance based on Coverage and Citation. Findings reveal that current systems lag behind human performance, with significant potential for future improvements. Read more
Can Long-Context Language Models Subsume Retrieval, RAG, SQL, and More? - This paper examines whether long-context language models (LCLMs) can take over tasks traditionally managed by retrieval systems and databases. It introduces the LOFT benchmark to evaluate LCLMs' performance on tasks that require processing contexts up to millions of tokens. The study finds that LCLMs can compete with state-of-the-art retrieval and RAG systems, though challenges persist in compositional reasoning. The research highlights the significance of effective prompting strategies and suggests that as context lengths increase, LCLMs may replace existing paradigms for tackling novel tasks. Read more
PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makers - This paper presents PlanRAG, a novel framework that enhances decision-making capabilities in large language models (LLMs) by integrating a plan-then-retrieval approach. The framework defines a new task, Decision QA, which involves planning, data retrieval, and answering questions based on business rules and databases. The authors propose the Decision QA benchmark (DQA) with scenarios designed from video games to simulate real-world decision-making situations. PlanRAG is shown to significantly outperform existing iterative RAG methods in both scenarios, emphasizing the importance of structured decision-making processes in LLM applications. Read more
From RAGs to Rich Parameters: Probing How Language Models Utilize External Knowledge over Parametric Information for Factual Queries - This paper investigates the mechanisms by which language models leverage external knowledge compared to their parametric information when addressing factual queries. The authors explore the efficacy of retrieval-augmented generation (RAG) techniques in conjunction with the models' internal parameters. Through extensive experiments, the study highlights the importance of balancing parametric information with external data to enhance response accuracy. The findings contribute to understanding the optimal configurations for integrating retrieval and parametric knowledge in language models. Read more
GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning - This paper presents GNN-RAG, a framework that integrates the language understanding capabilities of large language models (LLMs) with the reasoning power of graph neural networks (GNNs) for question answering over knowledge graphs (KGs). GNN-RAG processes a dense subgraph of the KG to retrieve answer candidates and extracts reasoning paths that connect entities, which are then verbalized for LLM input. Experimental results show GNN-RAG achieves state-of-the-art performance on knowledge graph question answering benchmarks, outperforming existing RAG methods. Read more
RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing - This survey paper explores the integration of external information retrieval with large language models (LLMs) to enhance performance across various natural language processing (NLP) tasks. It provides a comprehensive overview of Retrieval-Augmented Language Models (RALMs), including Retrieval-Augmented Generation (RAG) and Retrieval-Augmented Understanding (RAU). The paper examines essential components of RALMs, such as Retrievers, Language Models, and Augmentations, and their interactions that lead to diverse model structures. It highlights the utility of RALMs in tasks ranging from translation to dialogue systems and knowledge-intensive applications. The survey discusses various evaluation methods for RALMs, focusing on robustness, accuracy, and relevance, while acknowledging limitations in retrieval quality and computational efficiency. The paper concludes with future research direction
The Survey of Retrieval-Augmented Text Generation in Large Language Models - This paper explores Retrieval-Augmented Generation (RAG), which integrates retrieval methods with deep learning to overcome the static limitations of large language models (LLMs) by allowing the dynamic use of up-to-date external information. Focusing on the text domain, RAG enhances the accuracy and reliability of LLM outputs by leveraging real-world data. The authors categorize the RAG paradigm into four areas: pre-retrieval, retrieval, post-retrieval, and generation, providing insights into its mechanics and progression through key studies. The paper also discusses evaluation methods, challenges in the field, and suggests future research directions. By organizing existing research and clarifying technological foundations, the study aims to enhance the adaptability and applications of LLMs. Read more
RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generation - This paper introduces the Retrieval Augmented Thoughts (RAT) framework, which enhances reasoning and generation abilities in large language models (LLMs) for long-horizon tasks. RAT improves performance by revising thought steps iteratively with retrieved information relevant to the task, significantly reducing hallucination. Applied to models like GPT-3.5 and GPT-4, RAT shows substantial performance gains across various tasks including:
1. Code generation
2. Mathematical reasoning
3. Creative writing
The findings underscore the effectiveness of combining retrieval with context-aware reasoning in LLMs. Read more
Retrieval-Augmented Generation for AI-Generated Content: A Survey - This survey provides a comprehensive overview of Retrieval-Augmented Generation (RAG) techniques specifically tailored for AI-generated content. The authors discuss various methodologies, applications, and advancements in integrating retrieval mechanisms with generative models to enhance content quality and relevance. The paper explores challenges in current RAG systems and offers insights into future research directions, focusing on how RAG can be utilized to improve the generation of diverse content types, including text, images, and multimodal outputs. Read more
Reducing Hallucination in Structured Outputs via Retrieval-Augmented Generation - This paper addresses the issue of hallucination in Generative AI (GenAI) applications, specifically in the context of structured output tasks such as converting natural language requirements into workflows. The authors present a system that utilizes Retrieval-Augmented Generation (RAG) to enhance the quality of structured outputs, significantly reducing hallucination and allowing for better generalization of large language models (LLMs) to out-of-domain settings. The study demonstrates that employing a small, well-trained retriever can decrease the size of the LLM without sacrificing performance, making LLM-based systems more resource-efficient. Read more
Adaptive Filtering for RAG Systems strong> This paper presents E2E-AFG, an end-to-end model with adaptive filtering for RAG systems. Adaptive filtering enables the model to focus more effectively on relevant content while reducing the influence of irrelevant information and generating accurate answers. The proposed E2E-AFG consistently outperforms baseline models across all tasks. Read more
RAGViz Tool This paper presents RAGViz, a RAG diagnosis tool that visualizes the attentiveness of the generated tokens in retrieved documents. The RAGViz tool includes a built-in user interface, retrieval index, and Large Language Model (LLM) backbone. RAGViz provides two main functionalities: (1) token and document-level attention visualization, and (2) generation comparison upon context document addition and removal. RAGViz operates efficiently with a median query time of about 5 seconds on a moderate GPU node. Read more

Curated list of RAG Papers

Patrick Lewis @ NLP course UCL COMP0087