Beyond Simple Chatbots: Building Context-Aware AI with LangChain & Node.js

In the early days of generative AI applications, developers built simple "prompt-in, text-out" interfaces. While impressive for one-off completions, stateless chatbots quickly disappoint users when deployed in production. Without persistent conversational memory or access to proprietary organizational data, standard Large Language Models (LLMs) suffer from amnesia, hallucinate answers to private questions, and lose conversational thread context after a single turn.

Building truly intelligent, context-aware AI agents requires two architectural capabilities:

Conversational Memory: Retaining multi-turn dialog history across distributed user sessions without blowing through LLM token context limits.
Retrieval-Augmented Generation (RAG): Grounding LLM responses in real-time, domain-specific proprietary knowledge via vector embeddings.

LangChain for JavaScript/TypeScript (@langchain/core, @langchain/openai) provides an enterprise framework for constructing these pipelines. In this comprehensive guide, we construct a production-ready conversational AI system in Node.js, featuring Redis-backed persistent memory, LangChain Expression Language (LCEL), and Vector RAG.

SQL

+-------------------------------------------------------------------------------+
|                       Context-Aware AI Architecture                           |
+-------------------------------------------------------------------------------+
| User Prompt ---> [Session Token: Redis History] (Pulls last K turns)          |
|             ---> [Vector Embedding Search]      (Pulls domain context chunks) |
|             ---> [LCEL Runnable Pipeline]       (Merges prompt + history + RAG|
|             ---> [Streaming LLM: OpenAI / Claude] ──> Streamed Token Response |
+-------------------------------------------------------------------------------+

MERMAID

graph TD
    User([User Prompt]) --> Gateway[API Gateway / Node.js]
    Gateway --> Session[RedisChatMessageHistory]
    Gateway --> Embed[Embed Prompt via text-embedding-3-small]
    
    Session -->|Retrieve Last K Messages| LCEL[LCEL Runnable Pipeline]
    Embed -->|Cosine Similarity Query| Vector[(Vector Store: Pinecone / Chroma)]
    Vector -->|Top 3 Document Chunks| LCEL
    
    LCEL --> Prompt[Formatted System Prompt]
    Prompt --> LLM[ChatOpenAI: gpt-4o]
    LLM --> Stream[Stream Chunks to Client]
    LLM -.->|Append Turn| Session

1. Conversational Memory: Managing Token Windows

Naively appending every chat message to the prompt eventually hits the LLM's maximum context window and dramatically inflates API billing costs.

Memory Strategies in LangChain:

BufferWindowMemory: Retains only the last $K$ conversational turns (e.g. last 6 messages), automatically discarding older interactions.
ConversationSummaryMemory: Uses an auxiliary, inexpensive LLM to maintain a continuously updated summary of older conversation history while retaining recent raw turns.
Distributed Redis Memory: Stores chat history in an external Redis cluster indexed by sessionId, allowing users to resume conversations seamlessly across horizontally scaled Node.js pods.

2. Production Implementation with LCEL & Redis Memory

Let us build an enterprise conversational RAG engine utilizing modern LangChain Expression Language (LCEL) and persistent Redis storage:

TYPESCRIPT

// src/services/agent.service.ts
import { ChatOpenAI, OpenAIEmbeddings } from '@langchain/openai';
import { MemoryVectorStore } from 'langchain/vectorstores/memory';
import { Document } from '@langchain/core/documents';
import {
  ChatPromptTemplate,
  MessagesPlaceholder,
} from '@langchain/core/prompts';
import {
  RunnableSequence,
  RunnablePassthrough,
} from '@langchain/core/runnables';
import { StringOutputParser } from '@langchain/core/output_parsers';
import { RedisChatMessageHistory } from '@langchain/community/stores/message/ioredis';
import { Redis } from 'ioredis';

// 1. Initialize Redis Client for Persistent Session History
const redisClient = new Redis(process.env.REDIS_URL || 'redis://127.0.0.1:6379');

// 2. Initialize Seed Knowledge Base & Vector Store
const embeddings = new OpenAIEmbeddings({
  model: 'text-embedding-3-small',
});

const vectorStore = new MemoryVectorStore(embeddings);

// Seed domain documentation into vector store
await vectorStore.addDocuments([
  new Document({
    pageContent: 'Enterprise deployment SLA requires 99.99% availability and Kubernetes multi-region failover.',
    metadata: { source: 'sla-policy.md' },
  }),
  new Document({
    pageContent: 'Billing subscriptions renew on the 1st of every month. Cancellations must be submitted 7 days in advance.',
    metadata: { source: 'billing-terms.md' },
  }),
]);

const retriever = vectorStore.asRetriever(2);

// 3. Construct Prompt with Context & History Placeholders
const prompt = ChatPromptTemplate.fromMessages([
  [
    'system',
    'You are an enterprise AI support engineer. Answer the user question using ONLY the provided context. If the answer is unknown, state that you do not have sufficient information.\n\nContext:\n{context}',
  ],
  new MessagesPlaceholder('chat_history'),
  ['human', '{question}'],
]);

const llm = new ChatOpenAI({
  model: 'gpt-4o',
  temperature: 0.2,
  streaming: true,
});

// Helper to format retrieved documents into a clean string
function formatDocuments(docs: Document[]): string {
  return docs.map((doc) => doc.pageContent).join('\n---\n');
}

// 4. Construct the LCEL Runnable Chain
const conversationalChain = RunnableSequence.from([
  RunnablePassthrough.assign({
    context: async (input: { question: string }) => {
      const docs = await retriever.getRelevantDocuments(input.question);
      return formatDocuments(docs);
    },
  }),
  prompt,
  llm,
  new StringOutputParser(),
]);

3. Serving Chat Requests with Session Persistence

TYPESCRIPT

// src/server.ts
import express, { Request, Response } from 'express';
import { conversationalChain, redisClient } from './services/agent.service';
import { RedisChatMessageHistory } from '@langchain/community/stores/message/ioredis';

const app = express();
app.use(express.json());

app.post('/api/chat', async (req: Request, res: Response) => {
  const { sessionId, message } = req.body;

  if (!sessionId || !message) {
    return res.status(400).json({ error: 'sessionId and message are required' });
  }

  // Retrieve user session history from Redis
  const messageHistory = new RedisChatMessageHistory({
    sessionId: `chat:session:${sessionId}`,
    sessionTTL: 86400 * 7, // Retain session for 7 days
    client: redisClient,
  });

  const previousMessages = await messageHistory.getMessages();

  // Execute conversational chain with RAG context
  const aiResponse = await conversationalChain.invoke({
    question: message,
    chat_history: previousMessages,
  });

  // Persist current conversation turn back to Redis
  await messageHistory.addUserMessage(message);
  await messageHistory.addAIMessage(aiResponse);

  return res.json({
    sessionId,
    response: aiResponse,
  });
});

app.listen(3000, () => {
  console.log('[AI Server] Agent listening on http://localhost:3000');
});

Memory Strategy Comparison Matrix

Strategy	Token Consumption	Context Retention	Latency Overhead	Multi-Pod Scalability
`BufferMemory`	Infinite (Grows linearly)	Complete raw text	Low initially, then High	None (In-memory)
`BufferWindowMemory` (K=6)	Capped (Predictable)	Last K interactions	Ultra-Low (<1ms)	None (In-memory)
`SummaryMemory`	Fixed	Semantic summary	High (Extra LLM call)	None (In-memory)
`RedisChatMessageHistory`	Capped with Windowing	7+ Days Persistent	~2ms (Network hop)	Infinite (Shared Redis)

Production Verification Checklist

Context Window Guards: Ensure chat history arrays are bounded using a sliding window or summary before injecting into prompts.
Redis TTL Configured: Validate that session keys have an explicit expiration (sessionTTL: 604800) to prevent Redis storage exhaustion.
Low Temperature for RAG: Set temperature: 0.1 - 0.2 on RAG agents to prevent factual hallucinations.
Strict System Prompts: Instruct the model explicitly to decline answering when the retrieved context does not contain the answer.
Token Streaming Enabled: Enable streaming: true with Server-Sent Events (SSE) to reduce perceived user latency from 3,000ms to under 200ms.

Beyond Simple Chatbots: Building Context-Aware AI with LangChain & Node.js