Workshop 3: ChatGPT with RAG (Retrieval-Augmented Generation)
Build Environmental Stewardship-Specific LLMs Using Retrieval Augmented Generation
Last updated
Build Environmental Stewardship-Specific LLMs Using Retrieval Augmented Generation
Last updated
Users gain a lot of insights and value through Generative AIs like ChatGPT and ClaudeAI.
But, the problem with ChatGPT is that it does not have up-to-date data and may be missing specialized data that is relevant to a specific industry. Try asking ChatGPT "What is the carbon emissions in the US for 2023?" and you will get ...
old version of ChatGPT I apologize for the confusion, but as an AI language model, I don’t have real-time data or browsing capabilities. My responses are based on information available up until September 2021. Therefore, I cannot provide you with the current carbon emissions rate in the U.S. new version of ChatGPT
I'm sorry, but I don't have real-time data, and my training only includes information up to January 2022. For the most up-to-date and accurate information on carbon emissions in the United States for 2022 and 2023, I recommend checking official sources such as the U.S. Environmental Protection Agency (EPA) or other relevant government agencies. They typically release annual reports and data on carbon emissions and other environmental indicators.
This is a problem that Retrieval Augmented Generation (RAG) can help solve. Retrieval augmented generation refers to a technique in natural language processing where a large external knowledge source is utilized to augment a language model's text generation capabilities.
RAG key ideas are:
Use a large repository of texts as an external knowledge source - this can be Wikipedia, product manuals, a company's internal docs etc.
Index these texts in some retrievable format - An FAQ index, Elasticsearch index, etc.
When the language model needs to generate text, first query the retrieval/search index to find relevant knowledge.
Retrieve the most relevant texts and provide to the language model as context along with the original prompt.
The language model leverages this relevant external knowledge to generate more informed, accurate and detailed text.
Some benefits of retrieval augmented generation:
Helps tackle language model's limitation of limited knowledge capacity.
Generates outputs grounded in facts rather than hallucinations.
Can generate content conditioned on specific external context.
Allows tapping into large external knowledge sources.
Improves accuracy and relevance of generated text.
So in summary, retrieval augmentation enhances language models by allowing them to leverage huge external knowledge sources dynamically during text generation. The retriever provides relevant context to the generator.
How does RAG help create more accurate answers and prevent Generative AI "Hallucinations"?
Retrieval augmented generation (RAG) helps prevent hallucinations in language model generated text by grounding the outputs in factual information retrieved from external knowledge sources. Here are some of the key ways RAG helps:
The retriever finds the most relevant texts from the knowledge source to provide factual context to the generator. This reduces the model's need to "hallucinate" or make up facts.
The generator is conditioned on real external texts retrieved dynamically at inference time. This steers the model towards factual accuracy vs imaginative fiction.
Using the retriever's relevance signal, the RAG model can focus on texts that have higher factual alignment with the original query/prompt.
The large external knowledge source acts as a memory bank of facts for the model, reducing need forguesswork.
RAG models are often fine-tuned jointly with the retriever, further optimizing the system forfactual grounding.
During training, RAG models learn to differentiate hallucinations from grounded facts based on alignment with retrieved evidence.
Ranking/scoring mechanisms can further bias RAG models to favor generated text that exhibits greater factual alignment.
Overall, by providing fast access to a vast repository of facts, the retriever component in RAG models anchors the text generation to reality rather than imagination. This dynamic grounding in external knowledge is the key mechanism that reduces unfaithful hallucinations.
Our team can work on building Environmental Stewardship-Specific LLMs based on open source software called Quivr.
Add Knowledge to Custom Knowledge Databases
By crawling website URL or adding files to Custome Knowledge Databases
Chat with Custom Knowledge databases
Fast and Efficient: Designed with speed and efficiency at its core. Quivr ensures rapid access to your data.
Secure: Your data, your control. Always.
OS Compatible: Ubuntu 22 or newer.
File Compatibility: Text, Markdown, PDF, Powerpoint, Excel, CSV, Word, Audio, Video
Open Source: Freedom is beautiful, and so is Quivr. Open source and free to use.
Public/Private: Share your brains with your users via a public link, or keep them private.
Marketplace: Share your brains with the world, or use other people's brains to boost your productivity.
Follow these instructions to get a copy of the project up and running on your local machine for development and testing purposes.
Before you begin, make sure you have the following tools and accounts installed and set up:
Docker
Docker Compose
A Supabase account with:
A new Supabase project
Supabase Project API key
Supabase Project URL
Follow these steps to install and set up the Quivr project:
Use one of the following commands to clone the Quivr repository:
If you don't have an SSH key set up:
Run the install_helper.sh script to automate the setup process. This script will help you set up your environment files and execute the necessary migrations. Ensure you have the following prerequisites installed:
If you prefer manual setup, you can follow the steps below.
Copy the environment files as follows:
Copy .backend_env.example to backend/.env
Copy .frontend_env.example to frontend/.env
Edit the backend/.env and frontend/.env files with the following information:
supabase_service_key: Found in your Supabase dashboard under Project Settings -> API (Use the anon public key from the Project API keys section).
JWT_SECRET_KEY: Found in your Supabase settings under Project Settings -> API -> JWT Settings -> JWT Secret.
NEXT_PUBLIC_BACKEND_URL: Set to localhost:5050 for Docker. Update if your backend is running on a different machine.
Run the migration.sh script to execute the migration scripts. You have two options:
Create all tables: For the first-time setup.
Run migrations: When updating your database.
You can also run the script on the Supabase database via the web interface (SQL Editor -> New query -> paste the script -> Run). All migration scripts can be found in the scripts folder.
If you're migrating from an old version of Quivr, run the scripts in the migration script to update your data in chronological order.
Run the following command to launch the application:
Quivr is a like a personal ChatGPT enriched with your own custom data. Quivr has a number of great features, including ability to crawl and add website or upload files like text, spreadsheets, pdfs and ppt to LLM database so you can get LLM generated answers on your own data.
You can find everything on the .
If needed, you can watch the installation process on YouTube .
If you have an SSH key set up or want to add it (guide ):
Open your web browser and navigate to .
