Skip to content

NLQL - Natural Language Query Language

NLQL (Natural Language Query Language) is a SQL-like query language designed for querying unstructured text, vector databases, and RAG (Retrieval-Augmented Generation) systems.

Overview

NLQL combines the deterministic logic of SQL with the semantic capabilities of NLP, providing a unified interface for:

  • Vector databases (ChromaDB, FAISS, Qdrant, etc.)
  • Text collections with metadata
  • Hybrid retrieval (semantic + keyword + metadata filtering)
  • RAG pipelines with flexible granularity control

Key Features

🎯 SQL-like Syntax for Text

SELECT SENTENCE
WHERE SIMILAR_TO("AI agents") > 0.8
  AND META("date") > "2024-01-01"
ORDER BY SIMILARITY DESC
LIMIT 5

📊 Flexible Granularity

Control the level of detail in your results:

  • DOCUMENT - Full documents
  • CHUNK - Pre-chunked segments (from vector DB)
  • SENTENCE - Individual sentences
  • SPAN(unit, window=N) - Sliding windows with context

🔌 Extensible Architecture

  • Custom operators: Register domain-specific operators
  • Custom functions: Add your own query functions
  • Custom types: Define metadata field types
  • Custom splitters: Implement language-specific text splitting
  • Custom embeddings: Use any embedding model

âš¡ Smart Query Routing

NLQL automatically optimizes queries by:

  1. Push-down: Delegates filters to vector databases when possible
  2. In-memory: Executes complex logic on retrieved results
  3. Hybrid: Combines both strategies for optimal performance

Quick Start

Installation

pip install python-nlql

For vector database support:

# ChromaDB
pip install python-nlql[chroma]

# FAISS
pip install python-nlql[faiss]

# Qdrant
pip install python-nlql[qdrant]

# All adapters
pip install python-nlql[all]

Basic Usage

from nlql import NLQL
from nlql.adapters import MemoryAdapter

# Create adapter and add data
adapter = MemoryAdapter()
adapter.add_text("AI agents are autonomous systems")
adapter.add_text("Machine learning powers modern AI")

# Initialize NLQL with explicit adapter
nlql = NLQL(adapter=adapter)

# Execute query
results = nlql.execute("""
    SELECT CHUNK
    WHERE CONTAINS("AI")
    LIMIT 10
""")

# Process results
for result in results:
    print(result.content)
    print(result.metadata)

Architecture

NLQL uses a three-stage execution model:

  1. Parsing: Lark-based parser converts NLQL to AST
  2. Routing: Analyzes query and determines push-down vs in-memory execution
  3. Reshaping: Applies granularity transformations (SENTENCE, SPAN, etc.)

See Architecture for detailed information.

Next Steps

License

MIT License - see LICENSE file for details.