AI, RAG and the importance of Memory
AI Agents Need Memory
In our last article, we explained how an agentic AI Sales Development Representative can move the needle of your outbound lead generation motions.
That’s easier said than done. Sales teams can’t scale outbound lead generation if they have to constantly fact-check the AI agent’s work. They need to be able to trust its output. Which means that the AI agent needs to work with reliable, up-to-date company and market knowledge. That’s where the Retrieval-Augmented Generation (RAG) system comes in.
RAG Gives AI Agents Memory
Retrieval-Augmented Generation (RAG) is a technique that enhances large language models by connecting them to external knowledge sources to retrieve relevant information before generating responses, improving accuracy. A RAG hands the AI agent a briefing pack, created from curated, authoritative company sources such as CRM data, product documentation, market feeds, or internal knowledge bases. When the AI agent generates a response, it uses RAG-generated data to shape its output. The results are more aligned with company data, and prone to fewer hallucinations.
Internal Workings of RAG
A RAG system consists of three components:
- A Knowledge Base: The AI agent needs access to the right information: CRM records, product sheets, playbooks, or market data. A RAG system organizes the content in a vectorized database, to quickly retrieve the most relevant pieces when needed.
- A Retrieval Mechanism: When a user or another AI agent makes a request, the RAG system searches through the knowledge base and retrieves the most relevant elements of data that best answers the request.
- Prompt Augmentation: The retrieved content is added to the AI’s prompt. Instead of answering with only its general training, the model now has the most up-to-date, company-specific information to work with.
How to Build A RAG System Step by Step
1. Identify the Knowledge Base
The starting point is defining the scope of knowledge your RAG should access. This usually means gathering relevant documents and data sources that matter for your use case. Things like playbooks, product sheets, policies, or past case studies.
2. Build the Knowledge Base
A RAG system ingests the knowledge base to work with it. A good RAG system doesn’t just ingest an entire document blindly; it breaks it into smaller pieces it can search through. Each piece of content is converted into what’s called an embedding vector: a list of numbers that represents its meaning.
After vectorization, done with tools like Vertex AI, two passages with similar meaning will have similar vectors, while unrelated passages look very different. This lets the system match a user’s question with the most relevant pieces of information, even if the wording is different.
3. Retrieve Relevant Content
With embeddings in place, the system can search for relevant context based on meaning. It can understand that two phrases with different wording but the same intent (like “CEO” and “Chief Executive Officer”) refer to the same concept. The embeddings make the retrieval much more accurate and reliable, helping the system consistently surface the most relevant information.
Crucially, the system should also return the source along with its answer, so a human can double-check the facts if needed.
4. Supply Context to the LLM
Once the RAG system retrieves the right information, it passes it to the LLM. This adjusts the prompt, which now includes the user’s query, plus data supplied by the RAG system. The LLM then generates a grounded answer using that context.
5. Assure Quality Output
RAG systems are only as strong as the data they rely on. Without discipline, the knowledge base can quickly become messy, outdated, or inconsistent.
RAG systems need to be maintained regularly to remain trustworthy. Without maintenance, the system devolves to “garbage in, garbage out”. Avoiding this means you should think about:
- Data governance: ensuring ownership and accountability for what goes into the knowledge base.
- Quality control: setting standards for accuracy, attribution, and alignment with brand voice. For instance
- Golden set tests are pre-defined questions with known correct answers to make sure the system performs reliably
- Humans in the loop: have people double-check critical outputs, especially in early stages or high-stakes contexts
- Update the knowledge base: the knowledge base must be refreshed when its content changes, or the system will give outdated answers.
Examples of RAG Use Cases in Sales AI Agents
RAG systems ground sales agentic AI workflows in trusted company and market data.
- Scanner Agent: The Scanner agent identifies new accounts that fit the ICP and builds the top of the funnel. Using a RAG system, the Scanner agent can query a knowledge base built from CRM history, curated account lists, and up-to-date market news to ensure that only relevant, timely leads are surfaced.
- Enricher Agent: The Enricher agent adds data to leads and contacts. Using a RAG system, the Enricher agent draws from knowledge bases like funding databases, leadership trackers, and internal case studies. This creates enriched CRM records that contain accurate, verified details that reps can trust.
- Nurture Agent: The Nurture agent keeps conversations alive. It warms up cold leads while also developing existing ones through consistent, personalized touchpoints. Using a RAG system, the Nurture agent can access a knowledge base containing past conversations, product updates, campaign materials, and relevant company news. By drawing on this mix of historical and real-time context, it delivers messages that are both factually accurate and tailored to each lead or account—strengthening relationships while avoiding the pitfalls of guesswork.
If you’ve held back on using AI agents in your sales motions because you were concerned about inaccuracies or hallucinations, RAG is a great way to get them to behave better.
Have you started incorporating one or more RAG in your IT architecture to start leveraging company data into your AI workflows? Tell me all about it!
By Nicolas Mialaret and Maurits Van Roozendaal
ccc
Figure out AI for your business Book a call with Asteri!
Mores articles on Revenue
Receive exclusive Benchmarks and Research directly in your inbox Subscribe to our ASTERI+
Monthly Summary of ASTERI Insights and more Subscribe on Linkedin
