文獻統整文章

• A Practitioners Guide to Retrieval Augmented Generation (RAG) by Cameron R. Wolfe
  • What is Retrieval Augmented Generation?
    • The Structure of a RAG Pipeline
    • The Benefits of RAG
  • From the Origins of RAG to Modern Usage
  • Practical Tips for RAG Applications

經典文獻

• Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., ... & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33, 9459-9474. (https://proceedings.neurips.cc/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf)
  • 這篇文章是RAG的起源
• Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., ... & Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997. (https://arxiv.org/pdf/2312.10997)
  • 提到Naive RAG、Advanced RAG、Modular RAG
  • LLM RAG Paradigms: Naive RAG, Advanced RAG & Modular RAG
    • 本文為以上論文的摘要

統整性文獻

• Li, H., Su, Y., Cai, D., Wang, Y., & Liu, L. (2022). A survey on retrieval-augmented text generation. arXiv preprint arXiv:2202.01110. (https://arxiv.org/pdf/2202.01110)
  • 這篇論文整理了RAG的應用領域
    • Dialogue Response Generation
    • Machine Translation
    • Other Tasks
• Huang, Y., & Huang, J. (2024). A Survey on Retrieval-Augmented Text Generation for Large Language Models. arXiv preprint arXiv:2404.10981. (https://arxiv.org/pdf/2404.10981)
  • 這篇文章整理了RAG的相關技術
    • Pre-Retrieval
      • Indexing
      • Query Manipulation
      • Data Modification
    • Retrieval
      • Search & Ranking
      • Retrieval Strategy
    • Post-Retrieval
      • Re-Ranking
      • Filtering
    • Generation
      • Enhancing
      • Customization
    • Evaluation in RAG
      • Retrieval-based Aspect
      • Generation-based Aspect
• Wu, S., Xiong, Y., Cui, Y., Wu, H., Chen, C., Yuan, Y., ... & Xue, C. J. (2024). Retrieval-Augmented Generation for Natural Language Processing: A Survey. arXiv preprint arXiv:2407.13193. (https://arxiv.org/pdf/2407.13193)
  • 這篇文章整理了RAG的相關技術
    • RETRIEVER
      • Building the Retriever
      • Querying the Retriever
    • RETRIEVAL FUSIONS
      • Query-based Fusion
      • Logits-based Fusion
      • Latent Fusion
    • RAG TRAINING AND DATASTORE UPDATE
    • RAG EVALUATION AND BENCHMARK
    • TASKS
      • Language Modeling
      • Machine Translation
      • Text Summarization
      • Question Answering
      • Information Extraction
      • Text Classification
      • Dialogue Systems
• Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., ... & Cui, B. (2024). Retrieval-augmented generation for ai-generated content: A survey. arXiv preprint arXiv:2402.19473. (https://arxiv.org/pdf/2402.19473)
  • METHODOLOGIES
    • RAG Foundations
      • Query-based RAG
      • Latent Representation-based RAG
      • Logit-based RAG
      • Speculative RAG
    • RAG Enhancements
      • Input Enhancement
      • Retriever Enhancement
      • Generator Enhancement
      • Result Enhancement
      • RAG Pipeline Enhancement
  • APPLICATIONS
    • RAG for Text
    • RAG for Code
    • RAG for Knowledge
    • RAG for Image
    • RAG for Video
    • RAG for Audio
    • RAG for 3D
    • RAG for Science
  • BENCHMARK
• Wang, X., Wang, Z., Gao, X., Zhang, F., Wu, Y., Xu, Z., ... & Huang, X. (2024). Searching for best practices in retrieval-augmented generation. arXiv preprint arXiv:2407.01219. (https://arxiv.org/pdf/2407.01219)
  • 這個研究試圖在這麼多的RAG技術中找到最佳組合
    • 在Retrieval Module中，Hybrid with HyDE的效果最好，但是，處理時間也是非常的長 (11.71秒)，所以，要看系統的效率要求來取捨
    • Reranking Module有其必要，其中MonoT5效果最好
    • Repacking Module也是可以提升效能
    • Summarization Module也是有其必要，一樣的，也會增加處理時間，但是，在context有限的情況下，Summarization Module是相當重要的
• Barnett, S., Kurniawan, S., Thudumu, S., Brannelly, Z., & Abdelrazek, M. (2024, April). Seven failure points when engineering a retrieval augmented generation system. In Proceedings of the IEEE/ACM 3rd International Conference on AI Engineering-Software Engineering for AI (pp. 194-199). (https://dl.acm.org/doi/pdf/10.1145/3644815.3644945)
  • Failure points

Adaptive RAG

• Jeong, S., Baek, J., Cho, S., Hwang, S. J., & Park, J. C. (2024). Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity. arXiv preprint arXiv:2403.14403. (https://arxiv.org/pdf/2403.14403)

Corrective RAG

• Yan, S. Q., Gu, J. C., Zhu, Y., & Ling, Z. H. (2024). Corrective retrieval augmented generation. arXiv preprint arXiv:2401.15884. (https://arxiv.org/pdf/2401.15884)

Self RAG

• Asai, A., Wu, Z., Wang, Y., Sil, A., & Hajishirzi, H. (2023). Self-rag: Self-reflective retrieval augmented generation. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following. (https://openreview.net/pdf?id=jbNjgmE0OP)
• Asai, A., Wu, Z., Wang, Y., Sil, A., & Hajishirzi, H. (2023). Self-rag: Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310.11511. (https://arxiv.org/pdf/2310.11511) (同樣內容放在arXiv上)
• Chan, C. M., Xu, C., Yuan, R., Luo, H., Xue, W., Guo, Y., & Fu, J. (2024). Rq-rag: Learning to refine queries for retrieval augmented generation. arXiv preprint arXiv:2404.00610. (https://arxiv.org/pdf/2404.00610) (Refine Query for Retrieval Augmented Generation, RQ-RAG)