rag-solution/services/rag/langchain/QWEN.md

RAG Solution with Langchain

Project Overview

This is a Retrieval-Augmented Generation (RAG) solution built using the Langchain framework. The project is designed to load documents from a data directory, store them in a vector database (Qdrant), and enable semantic search and chat capabilities using local LLMs via Ollama.

The project follows a phased development approach with CLI entry points for different functionalities like document loading, retrieval, and chat.

Key Technologies:

• Framework: Langchain
• Vector Storage: Qdrant
• Embeddings: Ollama (with fallback option for OpenAI via OpenRouter)
• Chat Models: Ollama
• Data Directory: ./../../../data (relative to project root)
• Virtual Environment: Python venv in venv/ directory

Project Structure

rag-solution/services/rag/langchain/
├── .env.dist          # Environment variable template
├── .gitignore         # Git ignore rules
├── app.py             # Main application file (currently empty)
├── cli.py             # CLI entrypoint with click library
├── PLANNING.md        # Development roadmap and phases
├── QWEN.md            # Current file - project context
├── requirements.txt   # Python dependencies
└── venv/              # Virtual environment

Dependencies

The project relies on several key libraries:

• langchain and related ecosystem (langchain-community, langchain-core, langchain-ollama)
• langgraph for workflow management
• qdrant-client for vector storage (to be installed)
• ollama for local LLM interaction
• click for CLI interface
• loguru for logging (to be installed per requirements)
• python-dotenv for environment management

Development Phases

The project is organized into 6 development phases as outlined in PLANNING.md:

Phase 1: CLI Entrypoint

• Virtual environment setup
• Create CLI with click library
• Implement "ping" command that outputs "pong"

Phase 2: Framework Installation & Data Analysis

• Install Langchain as base RAG framework
• Analyze data folder extensions and create EXTENSIONS.md
• Install required loader libraries
• Configure environment variables

Phase 3: Vector Storage Setup

• Install Qdrant client library
• Create vector_storage.py for initialization
• Configure Ollama embeddings using OLLAMA_EMBEDDING_MODEL
• Prepare OpenAI fallback (commented)

Phase 4: Document Loading Module

• Create enrichment.py for loading documents to vector storage
• Implement text splitting strategies
• Add document tracking to prevent re-processing
• Integrate with CLI

Phase 5: Retrieval Feature

• Create retrieval.py for querying vector storage
• Implement metadata retrieval (filename, page, section, etc.)

Phase 6: Chat Agent

• Create agent.py with Ollama-powered chat agent
• Integrate with retrieval functionality
• Add CLI command for chat interaction

Environment Configuration

The project uses environment variables for configuration:

OLLAMA_EMBEDDING_MODEL=MODEL  # Name of the Ollama model for embeddings
OLLAMA_CHAT_MODEL=MODEL       # Name of the Ollama model for chat

Building and Running

Since the project is in early development stages, the following steps are planned:

1. Setup Virtual Environment:

   python -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate
   pip install -r requirements.txt
   

2. Install Missing Dependencies (as development progresses):

   pip install loguru qdrant-client  # Examples of needed libraries
   

3. Configure Environment:

   cp .env.dist .env
   # Edit .env with appropriate values
   

4. Run CLI Commands:

   python cli.py ping
   

Development Conventions

• Use loguru for logging with rotation to logs/dev.log and stdout
• Follow Langchain best practices for RAG implementations
• Prioritize open-source solutions that don't require external API keys
• Implement proper error handling and document processing tracking
• Use modular code organization with separate files for different components

Current Status

The project is in early development phase. The virtual environment is set up and dependencies are defined, but the core functionality (CLI, document loading, vector storage, etc.) is yet to be implemented according to the planned phases.