Retrieval Augmented Generation (RAG)
Module 1: RAG Overview
- RAG is used to improve quality and accuracy of LLMs.
- LLM starts only off with puppet internet data and you want the model to answer based on proprietary data, RAG allows this by giving it access to such data.
- In that sense, the LLM answers data on what it was NOT TRAINED ON.
- An example of this is LLM searching the web to answer your questions.
- Core idea is to pair the classical search systems with the reasoning abilities of LLMs.
Note: This is exactly what I thought AI should be used for, I just didn't konw the word for it. - Concept of RAG isn't complex, but there way many ways to implement it and that can have a big impact on the accuracy. Note: Remember what we learnt in the paper by Hilton. Current AI isn't a leap in intelligence, just engineering and a lot resources.
- Large companies are using it to help customers answer questions about products and using it internally for employees.
- With the LLMs getting larger and holding a large context window and Agents(WHICH ARE FUCKING EXPENSIVE AND HAS THE POTENTIAL TO RUIN A COMPANY) makes writing RAG even more easier.
- RAG is also a useful step in the Agentic workflow.
- Agentic RAG: Systems that use multiple LLMs where each one handles a single part of a large workflow and has the agency to decide what data to retrieve.
- LLMs are already powerful. RAG further improves them by giving them new information that they don't have.
- Various questions require varying degrees of specialized information.
-
(Retrieval) (Generation) Collect Information → Reason & Respond.
- LLMs don't perform so great with very recent information or specialized knowledge it hasn't previously seen.
LLMs can't be expect on every topic. - LLMs benefit from the Retrieval phase also instead of the traditional Generation phase is the core idea behind RAG.
- When you ask a question you are hoping the question was in the training data ideally many times, unfortunately, it won't be because:
- Private Databases
- Hard to access Information.
- Real time data: LLMs are trained on past data and don't update automatically.
- How do you get the LLM access specific information?
Answer: Just put it in the prompt.
I am speechless this is so stupid! - RAG system makes an augmented prompt and sends this to LLM. This step involves gathering relevant information.
- Retriever: The components of the RAG system that handles retrieving information.
Manages knowledge base of trusted information.
Then, it finds and retrieves the relevant information to share with the LLM. - RAG→ All you're doing is improving/augmenting the way an LLM generates text by first retrieving information from a knowledge base.
- Application of RAG → Code Generation.
Even though there are a lot of code around, the best code for your use case requires a few things beforehand:- Project's Context: Classes, functions, definitions, and coding style.
- Use the codebase as the knowledge base.
- Improves code generation and Q&A.
- Application of RAG → Company Chatbots.
- Tailored to a company: Every company has its own products, policies, and communication style.
- Uses internal documents: Manuals, support guides, FAQs
- Ground answers in real context: Reduces generic or incorrect answers.
- Application of RAG → Specialized Knowledge(Healthcare, Legal Domains):
- High-impact Domains
- Uses specialized documents: Case files, journals, private data.
- Enables accurate, secure use: Supports precision and privacy needs
In this course they keep talking about how "private" and "classified" information that you don't want to share with the public, you can share with an LLM to get better answers out of it.
And do these people not realize that these information are being sent to the AI company who are literally buying all the storage in the world just to store and sell that data.
I will rate this course one point lower if they didn't bother talking about the security risk of sharing confidential information to a chatbot.
-ni6hant- Application of RAG: Search Engines:
- Search engines as retrievers
- AI summarizes the search results
- RAG with internet as a knowledge base.
- Personalized RAG:
- More software includes personal assistants.
- These tools need context.
- Your data is the knowledge base.