Skip to content

Adapters API Reference

BaseAdapter

Bases: ABC

Abstract base class for data source adapters.

Design Philosophy - Adapters are DATA RETRIEVERS, not QUERY PROCESSORS:

Adapters are responsible for: 1. Retrieving data from the underlying data source 2. Converting data to TextUnit objects (NLQL's internal format) 3. Declaring capabilities (semantic search, etc.)

Adapters are NOT responsible for: 1. Filtering results based on WHERE clauses (handled by Executor) 2. Sorting results based on ORDER BY (handled by Executor) 3. Limiting results based on LIMIT (handled by Executor) 4. Granularity transformations like SENTENCE/SPAN (handled by Executor)

This separation of concerns ensures: - Simple adapter implementation (just focus on data retrieval) - Consistent query semantics across all data sources - Easy testing and debugging - Low mental overhead for adapter developers

Example Implementations: - MemoryAdapter: Returns all chunks from in-memory list - ChromaAdapter (future): Executes semantic search, returns top-k results - FAISSAdapter (future): Executes vector search, returns neighbors - SQLAdapter (future): Executes SQL query, returns rows as TextUnits

Note: While adapters CAN apply optimizations (e.g., using QueryPlan.filters for database-level filtering), they should always return semantically correct results. The Executor will apply additional filtering as needed.

Source code in src/nlql/adapters/base.py 
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
class BaseAdapter(ABC):
    """Abstract base class for data source adapters.

    Design Philosophy - Adapters are DATA RETRIEVERS, not QUERY PROCESSORS:

    Adapters are responsible for:
    1. Retrieving data from the underlying data source
    2. Converting data to TextUnit objects (NLQL's internal format)
    3. Declaring capabilities (semantic search, etc.)

    Adapters are NOT responsible for:
    1. Filtering results based on WHERE clauses (handled by Executor)
    2. Sorting results based on ORDER BY (handled by Executor)
    3. Limiting results based on LIMIT (handled by Executor)
    4. Granularity transformations like SENTENCE/SPAN (handled by Executor)

    This separation of concerns ensures:
    - Simple adapter implementation (just focus on data retrieval)
    - Consistent query semantics across all data sources
    - Easy testing and debugging
    - Low mental overhead for adapter developers

    Example Implementations:
    - MemoryAdapter: Returns all chunks from in-memory list
    - ChromaAdapter (future): Executes semantic search, returns top-k results
    - FAISSAdapter (future): Executes vector search, returns neighbors
    - SQLAdapter (future): Executes SQL query, returns rows as TextUnits

    Note: While adapters CAN apply optimizations (e.g., using QueryPlan.filters
    for database-level filtering), they should always return semantically
    correct results. The Executor will apply additional filtering as needed.
    """

    @abstractmethod
    def query(self, plan: QueryPlan) -> list[TextUnit]:
        """Retrieve data from the data source.

        This method should focus ONLY on data retrieval, not on filtering,
        sorting, or limiting. The Executor will handle all query logic.

        Typical implementations:
        - MemoryAdapter: Return all chunks
        - VectorDBAdapter: Execute semantic search with plan.query_text,
          return top-k similar chunks
        - SQLAdapter: Execute SQL query, return all matching rows

        Args:
            plan: Query plan containing retrieval parameters
                 - plan.query_text: For semantic/vector search
                 - plan.filters: Optional optimization hints (can be ignored)
                 - plan.limit: Should be ignored (Executor handles limiting)

        Returns:
            List of TextUnit objects retrieved from the data source.
            Do NOT apply WHERE filtering, ORDER BY sorting, or LIMIT here.

        Raises:
            NLQLAdapterError: If data retrieval fails
        """
        pass

    @abstractmethod
    def supports_semantic_search(self) -> bool:
        """Check if this adapter supports semantic/vector similarity search.

        This indicates whether the adapter can handle plan.query_text for
        semantic search (e.g., finding similar documents using embeddings).

        Returns:
            True if the adapter can perform semantic search (e.g., vector DB)
            False if the adapter only returns raw data (e.g., MemoryAdapter)

        Examples:
            - MemoryAdapter: False (no embeddings)
            - ChromaAdapter: True (has vector search)
            - FAISSAdapter: True (has vector search)
        """
        pass

    @abstractmethod
    def supports_metadata_filter(self) -> bool:
        """Check if this adapter can optimize metadata filtering.

        This indicates whether the adapter can use plan.filters to optimize
        data retrieval (e.g., database-level WHERE clauses).

        Note: Returning False does NOT mean metadata filtering is unsupported.
        It just means the adapter doesn't optimize it - the Executor will
        handle all filtering in memory.

        Returns:
            True if the adapter can apply plan.filters for optimization
            False if the adapter ignores plan.filters (Executor handles it)

        Examples:
            - MemoryAdapter: False (returns all data, Executor filters)
            - ChromaAdapter: True (can use Chroma's where clause)
            - SQLAdapter: True (can use SQL WHERE clause)
        """
        pass

    def get_capabilities(self) -> dict[str, bool]:
        """Get a dictionary of adapter capabilities.

        Returns:
            Dictionary mapping capability names to boolean values
        """
        return {
            "semantic_search": self.supports_semantic_search(),
            "metadata_filter": self.supports_metadata_filter(),
        }

query(plan) abstractmethod

Retrieve data from the data source.

This method should focus ONLY on data retrieval, not on filtering, sorting, or limiting. The Executor will handle all query logic.

Typical implementations: - MemoryAdapter: Return all chunks - VectorDBAdapter: Execute semantic search with plan.query_text, return top-k similar chunks - SQLAdapter: Execute SQL query, return all matching rows

Parameters:

Name Type Description Default
plan QueryPlan

Query plan containing retrieval parameters - plan.query_text: For semantic/vector search - plan.filters: Optional optimization hints (can be ignored) - plan.limit: Should be ignored (Executor handles limiting)

 required

Returns:

Type Description
list[TextUnit]

List of TextUnit objects retrieved from the data source.
list[TextUnit]

Do NOT apply WHERE filtering, ORDER BY sorting, or LIMIT here.

Raises:

Type Description
NLQLAdapterError

If data retrieval fails
Source code in src/nlql/adapters/base.py 
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
@abstractmethod
def query(self, plan: QueryPlan) -> list[TextUnit]:
    """Retrieve data from the data source.

    This method should focus ONLY on data retrieval, not on filtering,
    sorting, or limiting. The Executor will handle all query logic.

    Typical implementations:
    - MemoryAdapter: Return all chunks
    - VectorDBAdapter: Execute semantic search with plan.query_text,
      return top-k similar chunks
    - SQLAdapter: Execute SQL query, return all matching rows

    Args:
        plan: Query plan containing retrieval parameters
             - plan.query_text: For semantic/vector search
             - plan.filters: Optional optimization hints (can be ignored)
             - plan.limit: Should be ignored (Executor handles limiting)

    Returns:
        List of TextUnit objects retrieved from the data source.
        Do NOT apply WHERE filtering, ORDER BY sorting, or LIMIT here.

    Raises:
        NLQLAdapterError: If data retrieval fails
    """
    pass

Check if this adapter supports semantic/vector similarity search.

This indicates whether the adapter can handle plan.query_text for semantic search (e.g., finding similar documents using embeddings).

Returns:

Type Description
bool

True if the adapter can perform semantic search (e.g., vector DB)
bool

False if the adapter only returns raw data (e.g., MemoryAdapter)

Examples:

  • MemoryAdapter: False (no embeddings)
  • ChromaAdapter: True (has vector search)
  • FAISSAdapter: True (has vector search)
Source code in src/nlql/adapters/base.py 
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
@abstractmethod
def supports_semantic_search(self) -> bool:
    """Check if this adapter supports semantic/vector similarity search.

    This indicates whether the adapter can handle plan.query_text for
    semantic search (e.g., finding similar documents using embeddings).

    Returns:
        True if the adapter can perform semantic search (e.g., vector DB)
        False if the adapter only returns raw data (e.g., MemoryAdapter)

    Examples:
        - MemoryAdapter: False (no embeddings)
        - ChromaAdapter: True (has vector search)
        - FAISSAdapter: True (has vector search)
    """
    pass

supports_metadata_filter() abstractmethod

Check if this adapter can optimize metadata filtering.

This indicates whether the adapter can use plan.filters to optimize data retrieval (e.g., database-level WHERE clauses).

Note: Returning False does NOT mean metadata filtering is unsupported. It just means the adapter doesn't optimize it - the Executor will handle all filtering in memory.

Returns:

Type Description
bool

True if the adapter can apply plan.filters for optimization
bool

False if the adapter ignores plan.filters (Executor handles it)

Examples:

  • MemoryAdapter: False (returns all data, Executor filters)
  • ChromaAdapter: True (can use Chroma's where clause)
  • SQLAdapter: True (can use SQL WHERE clause)
Source code in src/nlql/adapters/base.py 
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
@abstractmethod
def supports_metadata_filter(self) -> bool:
    """Check if this adapter can optimize metadata filtering.

    This indicates whether the adapter can use plan.filters to optimize
    data retrieval (e.g., database-level WHERE clauses).

    Note: Returning False does NOT mean metadata filtering is unsupported.
    It just means the adapter doesn't optimize it - the Executor will
    handle all filtering in memory.

    Returns:
        True if the adapter can apply plan.filters for optimization
        False if the adapter ignores plan.filters (Executor handles it)

    Examples:
        - MemoryAdapter: False (returns all data, Executor filters)
        - ChromaAdapter: True (can use Chroma's where clause)
        - SQLAdapter: True (can use SQL WHERE clause)
    """
    pass

get_capabilities()

Get a dictionary of adapter capabilities.

Returns:

Type Description
dict[str, bool]

Dictionary mapping capability names to boolean values
Source code in src/nlql/adapters/base.py 
151
152
153
154
155
156
157
158
159
160
def get_capabilities(self) -> dict[str, bool]:
    """Get a dictionary of adapter capabilities.

    Returns:
        Dictionary mapping capability names to boolean values
    """
    return {
        "semantic_search": self.supports_semantic_search(),
        "metadata_filter": self.supports_metadata_filter(),
    }

QueryPlan dataclass

Represents a query plan for data retrieval from an adapter.

QueryPlan is used to communicate retrieval requirements from the Executor to the Adapter. It contains ONLY the information needed for data retrieval, not for filtering, sorting, or limiting results.

Design Philosophy: - Adapters are responsible for DATA RETRIEVAL only - Executors are responsible for QUERY LOGIC (WHERE, ORDER BY, LIMIT) - QueryPlan bridges these two layers

Attributes:

Name Type Description
query_text str | None

Text query for semantic/vector search (e.g., for SIMILAR_TO) Only used by adapters that support semantic search.
filters dict[str, Any] | None

Simple key-value filters that CAN be pushed down to the adapter for optimization (e.g., metadata filters for vector databases). Currently unused - all filtering is done in the Executor.
limit int | None

NOT USED. Limiting is handled by the Executor after filtering. This field is kept for future optimization scenarios.
metadata dict[str, Any] | None

Additional adapter-specific parameters for advanced use cases.
Note

In the current implementation (Simple Mode): - query_text: Used for semantic search adapters - filters: None (all filtering in Executor) - limit: None (all limiting in Executor)

In future optimized implementations: - The Executor may analyze WHERE clauses and push down simple filters - The QueryRouter will determine what can be safely pushed down

Source code in src/nlql/adapters/base.py 
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
@dataclass
class QueryPlan:
    """Represents a query plan for data retrieval from an adapter.

    QueryPlan is used to communicate retrieval requirements from the Executor
    to the Adapter. It contains ONLY the information needed for data retrieval,
    not for filtering, sorting, or limiting results.

    Design Philosophy:
    - Adapters are responsible for DATA RETRIEVAL only
    - Executors are responsible for QUERY LOGIC (WHERE, ORDER BY, LIMIT)
    - QueryPlan bridges these two layers

    Attributes:
        query_text: Text query for semantic/vector search (e.g., for SIMILAR_TO)
                   Only used by adapters that support semantic search.
        filters: Simple key-value filters that CAN be pushed down to the adapter
                for optimization (e.g., metadata filters for vector databases).
                Currently unused - all filtering is done in the Executor.
        limit: NOT USED. Limiting is handled by the Executor after filtering.
              This field is kept for future optimization scenarios.
        metadata: Additional adapter-specific parameters for advanced use cases.

    Note:
        In the current implementation (Simple Mode):
        - query_text: Used for semantic search adapters
        - filters: None (all filtering in Executor)
        - limit: None (all limiting in Executor)

        In future optimized implementations:
        - The Executor may analyze WHERE clauses and push down simple filters
        - The QueryRouter will determine what can be safely pushed down
    """

    query_text: str | None = None
    filters: dict[str, Any] | None = None
    limit: int | None = None
    metadata: dict[str, Any] | None = None

MemoryAdapter

Bases: BaseAdapter

Simple in-memory adapter for testing and prototyping.

This adapter stores chunks in memory and performs simple filtering. It's useful for testing, demonstrations, and small datasets without requiring a vector database.

The adapter provides convenient methods for adding data: - add_chunk(): Add a single chunk with metadata - add_text(): Add text (automatically creates a chunk) - add_texts(): Batch add multiple texts - add_document(): Add a document with automatic chunking

Example

adapter = MemoryAdapter() adapter.add_text("AI agents are autonomous systems", {"topic": "AI"}) adapter.add_text("Machine learning powers modern AI", {"topic": "ML"})

from nlql import NLQL nlql = NLQL(adapter=adapter) results = nlql.execute("SELECT CHUNK LIMIT 10")

Source code in src/nlql/adapters/memory.py 
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
class MemoryAdapter(BaseAdapter):
    """Simple in-memory adapter for testing and prototyping.

    This adapter stores chunks in memory and performs simple filtering.
    It's useful for testing, demonstrations, and small datasets without
    requiring a vector database.

    The adapter provides convenient methods for adding data:
    - add_chunk(): Add a single chunk with metadata
    - add_text(): Add text (automatically creates a chunk)
    - add_texts(): Batch add multiple texts
    - add_document(): Add a document with automatic chunking

    Example:
        >>> adapter = MemoryAdapter()
        >>> adapter.add_text("AI agents are autonomous systems", {"topic": "AI"})
        >>> adapter.add_text("Machine learning powers modern AI", {"topic": "ML"})
        >>>
        >>> from nlql import NLQL
        >>> nlql = NLQL(adapter=adapter)
        >>> results = nlql.execute("SELECT CHUNK LIMIT 10")
    """

    def __init__(self) -> None:
        """Initialize an empty memory adapter.

        Use add_chunk(), add_text(), or add_document() to populate data.
        """
        self._chunks: list[Chunk] = []

    def query(self, plan: QueryPlan) -> list[TextUnit]:  # noqa: ARG002
        """Execute a simple in-memory query.

        Note: MemoryAdapter is a simple adapter that returns all chunks.
        Filtering, ordering, and limiting are handled by the Executor.

        In the future, if plan.filters or plan.query_text are provided,
        this method could apply optimizations, but for now it returns
        all data and lets the executor handle the rest.

        Args:
            plan: Query plan (currently unused for MemoryAdapter)

        Returns:
            List of all chunks
        """
        # For MemoryAdapter, we simply return all chunks
        # The executor will handle WHERE filtering, ORDER BY, and LIMIT
        return self._chunks.copy()

    def supports_semantic_search(self) -> bool:
        """Memory adapter does not support semantic search.

        Semantic search requires embeddings and similarity computation,
        which is not implemented in the basic MemoryAdapter.
        """
        return False

    def supports_metadata_filter(self) -> bool:
        """Memory adapter does not push down metadata filters.

        While metadata filtering is supported by NLQL, the MemoryAdapter
        returns all chunks and lets the Executor handle filtering.
        This method returns False to indicate no pushdown optimization.
        """
        return False

    def add_chunk(
        self,
        content: str,
        metadata: dict[str, Any] | None = None,
        chunk_id: str | None = None,
    ) -> str:
        """Add a single chunk to the memory store.

        Args:
            content: Chunk content
            metadata: Optional metadata dictionary
            chunk_id: Optional custom chunk ID. If not provided, auto-generates one.

        Returns:
            The chunk ID of the added chunk

        Example:
            >>> adapter = MemoryAdapter()
            >>> chunk_id = adapter.add_chunk(
            ...     "AI agents are autonomous",
            ...     metadata={"topic": "AI", "date": "2024-01-01"}
            ... )
        """
        if chunk_id is None:
            chunk_id = f"chunk_{len(self._chunks)}"

        self._chunks.append(
            Chunk(
                content=content,
                metadata=metadata or {},
                chunk_id=chunk_id,
                position=len(self._chunks),
            )
        )
        return chunk_id

    def add_text(self, text: str, metadata: dict[str, Any] | None = None) -> str:
        """Add a single text as a chunk.

        This is a convenience method equivalent to add_chunk().

        Args:
            text: Text content
            metadata: Optional metadata dictionary

        Returns:
            The chunk ID of the added chunk

        Example:
            >>> adapter = MemoryAdapter()
            >>> adapter.add_text("AI agents are autonomous systems")
            >>> adapter.add_text("Machine learning powers AI", {"topic": "ML"})
        """
        return self.add_chunk(text, metadata)

    def add_texts(
        self,
        texts: list[str],
        metadatas: list[dict[str, Any]] | None = None,
    ) -> list[str]:
        """Batch add multiple texts as chunks.

        Args:
            texts: List of text contents
            metadatas: Optional list of metadata dictionaries (must match length of texts)

        Returns:
            List of chunk IDs for the added chunks

        Raises:
            ValueError: If metadatas length doesn't match texts length

        Example:
            >>> adapter = MemoryAdapter()
            >>> texts = [
            ...     "AI agents are autonomous",
            ...     "Machine learning powers AI",
            ...     "NLP enables text understanding"
            ... ]
            >>> metadatas = [
            ...     {"topic": "AI"},
            ...     {"topic": "ML"},
            ...     {"topic": "NLP"}
            ... ]
            >>> chunk_ids = adapter.add_texts(texts, metadatas)
        """
        if metadatas is not None and len(metadatas) != len(texts):
            raise ValueError(
                f"metadatas length ({len(metadatas)}) must match texts length ({len(texts)})"
            )

        chunk_ids = []
        for i, text in enumerate(texts):
            metadata = metadatas[i] if metadatas else None
            chunk_id = self.add_text(text, metadata)
            chunk_ids.append(chunk_id)

        return chunk_ids

    def add_document(
        self,
        document: str,
        metadata: dict[str, Any] | None = None,
        chunk_size: int = 500,
        chunk_overlap: int = 50,
        separator: str = "\n\n",
    ) -> list[str]:
        """Add a document with automatic chunking.

        The document will be split into chunks based on the specified parameters.
        Each chunk will inherit the document's metadata with an additional
        'chunk_index' field.

        Args:
            document: Full document text
            metadata: Optional metadata for the document (inherited by all chunks)
            chunk_size: Target size for each chunk (in characters)
            chunk_overlap: Number of characters to overlap between chunks
            separator: Separator to use for splitting (default: paragraph breaks)

        Returns:
            List of chunk IDs for the created chunks

        Example:
            >>> adapter = MemoryAdapter()
            >>> long_text = "..." # Long document
            >>> chunk_ids = adapter.add_document(
            ...     long_text,
            ...     metadata={"source": "paper.pdf", "author": "Alice"},
            ...     chunk_size=500,
            ...     chunk_overlap=50
            ... )
        """
        # Simple chunking strategy: split by separator first, then by size
        chunks = self._chunk_text(document, chunk_size, chunk_overlap, separator)

        chunk_ids = []
        base_metadata = metadata or {}

        for i, chunk_text in enumerate(chunks):
            chunk_metadata = base_metadata.copy()
            chunk_metadata["chunk_index"] = i
            chunk_metadata["total_chunks"] = len(chunks)

            chunk_id = self.add_chunk(chunk_text, chunk_metadata)
            chunk_ids.append(chunk_id)

        return chunk_ids

    def _chunk_text(
        self,
        text: str,
        chunk_size: int,
        chunk_overlap: int,
        separator: str,
    ) -> list[str]:
        """Split text into chunks.

        Args:
            text: Text to split
            chunk_size: Target chunk size
            chunk_overlap: Overlap between chunks
            separator: Separator for initial split

        Returns:
            List of text chunks
        """
        # First split by separator (e.g., paragraphs)
        segments = text.split(separator)

        chunks = []
        current_chunk = ""

        for segment in segments:
            segment = segment.strip()
            if not segment:
                continue

            # If adding this segment would exceed chunk_size, save current chunk
            if current_chunk and len(current_chunk) + len(segment) > chunk_size:
                chunks.append(current_chunk)

                # Start new chunk with overlap
                if chunk_overlap > 0 and len(current_chunk) > chunk_overlap:
                    current_chunk = current_chunk[-chunk_overlap:] + " " + segment
                else:
                    current_chunk = segment
            else:
                # Add to current chunk
                if current_chunk:
                    current_chunk += " " + segment
                else:
                    current_chunk = segment

        # Add the last chunk
        if current_chunk:
            chunks.append(current_chunk)

        return chunks

    def clear(self) -> None:
        """Clear all chunks from the adapter.

        Example:
            >>> adapter = MemoryAdapter()
            >>> adapter.add_text("Some text")
            >>> adapter.clear()
            >>> len(adapter) == 0
            True
        """
        self._chunks.clear()

    def __len__(self) -> int:
        """Get the number of chunks in the adapter."""
        return len(self._chunks)

    def __repr__(self) -> str:
        """String representation of the adapter."""
        return f"MemoryAdapter(chunks={len(self._chunks)})"

__init__()

Initialize an empty memory adapter.

Use add_chunk(), add_text(), or add_document() to populate data.

Source code in src/nlql/adapters/memory.py 
33
34
35
36
37
38
def __init__(self) -> None:
    """Initialize an empty memory adapter.

    Use add_chunk(), add_text(), or add_document() to populate data.
    """
    self._chunks: list[Chunk] = []

query(plan)

Execute a simple in-memory query.

Note: MemoryAdapter is a simple adapter that returns all chunks. Filtering, ordering, and limiting are handled by the Executor.

In the future, if plan.filters or plan.query_text are provided, this method could apply optimizations, but for now it returns all data and lets the executor handle the rest.

Parameters:

Name Type Description Default
plan QueryPlan

Query plan (currently unused for MemoryAdapter)

 required

Returns:

Type Description
list[TextUnit]

List of all chunks
Source code in src/nlql/adapters/memory.py 
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
def query(self, plan: QueryPlan) -> list[TextUnit]:  # noqa: ARG002
    """Execute a simple in-memory query.

    Note: MemoryAdapter is a simple adapter that returns all chunks.
    Filtering, ordering, and limiting are handled by the Executor.

    In the future, if plan.filters or plan.query_text are provided,
    this method could apply optimizations, but for now it returns
    all data and lets the executor handle the rest.

    Args:
        plan: Query plan (currently unused for MemoryAdapter)

    Returns:
        List of all chunks
    """
    # For MemoryAdapter, we simply return all chunks
    # The executor will handle WHERE filtering, ORDER BY, and LIMIT
    return self._chunks.copy()

Memory adapter does not support semantic search.

Semantic search requires embeddings and similarity computation, which is not implemented in the basic MemoryAdapter.

Source code in src/nlql/adapters/memory.py 
60
61
62
63
64
65
66
def supports_semantic_search(self) -> bool:
    """Memory adapter does not support semantic search.

    Semantic search requires embeddings and similarity computation,
    which is not implemented in the basic MemoryAdapter.
    """
    return False

supports_metadata_filter()

Memory adapter does not push down metadata filters.

While metadata filtering is supported by NLQL, the MemoryAdapter returns all chunks and lets the Executor handle filtering. This method returns False to indicate no pushdown optimization.

Source code in src/nlql/adapters/memory.py 
68
69
70
71
72
73
74
75
def supports_metadata_filter(self) -> bool:
    """Memory adapter does not push down metadata filters.

    While metadata filtering is supported by NLQL, the MemoryAdapter
    returns all chunks and lets the Executor handle filtering.
    This method returns False to indicate no pushdown optimization.
    """
    return False

add_chunk(content, metadata=None, chunk_id=None)

Add a single chunk to the memory store.

Parameters:

Name Type Description Default
content str

Chunk content

 required
metadata dict[str, Any] | None

Optional metadata dictionary

 None
chunk_id str | None

Optional custom chunk ID. If not provided, auto-generates one.

 None

Returns:

Type Description
str

The chunk ID of the added chunk
Example

adapter = MemoryAdapter() chunk_id = adapter.add_chunk( ... "AI agents are autonomous", ... metadata={"topic": "AI", "date": "2024-01-01"} ... )

Source code in src/nlql/adapters/memory.py 
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
def add_chunk(
    self,
    content: str,
    metadata: dict[str, Any] | None = None,
    chunk_id: str | None = None,
) -> str:
    """Add a single chunk to the memory store.

    Args:
        content: Chunk content
        metadata: Optional metadata dictionary
        chunk_id: Optional custom chunk ID. If not provided, auto-generates one.

    Returns:
        The chunk ID of the added chunk

    Example:
        >>> adapter = MemoryAdapter()
        >>> chunk_id = adapter.add_chunk(
        ...     "AI agents are autonomous",
        ...     metadata={"topic": "AI", "date": "2024-01-01"}
        ... )
    """
    if chunk_id is None:
        chunk_id = f"chunk_{len(self._chunks)}"

    self._chunks.append(
        Chunk(
            content=content,
            metadata=metadata or {},
            chunk_id=chunk_id,
            position=len(self._chunks),
        )
    )
    return chunk_id

add_text(text, metadata=None)

Add a single text as a chunk.

This is a convenience method equivalent to add_chunk().

Parameters:

Name Type Description Default
text str

Text content

 required
metadata dict[str, Any] | None

Optional metadata dictionary

 None

Returns:

Type Description
str

The chunk ID of the added chunk
Example

adapter = MemoryAdapter() adapter.add_text("AI agents are autonomous systems") adapter.add_text("Machine learning powers AI", {"topic": "ML"})

Source code in src/nlql/adapters/memory.py 
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
def add_text(self, text: str, metadata: dict[str, Any] | None = None) -> str:
    """Add a single text as a chunk.

    This is a convenience method equivalent to add_chunk().

    Args:
        text: Text content
        metadata: Optional metadata dictionary

    Returns:
        The chunk ID of the added chunk

    Example:
        >>> adapter = MemoryAdapter()
        >>> adapter.add_text("AI agents are autonomous systems")
        >>> adapter.add_text("Machine learning powers AI", {"topic": "ML"})
    """
    return self.add_chunk(text, metadata)

add_texts(texts, metadatas=None)

Batch add multiple texts as chunks.

Parameters:

Name Type Description Default
texts list[str]

List of text contents

 required
metadatas list[dict[str, Any]] | None

Optional list of metadata dictionaries (must match length of texts)

 None

Returns:

Type Description
list[str]

List of chunk IDs for the added chunks

Raises:

Type Description
ValueError

If metadatas length doesn't match texts length
Example

adapter = MemoryAdapter() texts = [ ... "AI agents are autonomous", ... "Machine learning powers AI", ... "NLP enables text understanding" ... ] metadatas = [ ... {"topic": "AI"}, ... {"topic": "ML"}, ... {"topic": "NLP"} ... ] chunk_ids = adapter.add_texts(texts, metadatas)

Source code in src/nlql/adapters/memory.py 
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
def add_texts(
    self,
    texts: list[str],
    metadatas: list[dict[str, Any]] | None = None,
) -> list[str]:
    """Batch add multiple texts as chunks.

    Args:
        texts: List of text contents
        metadatas: Optional list of metadata dictionaries (must match length of texts)

    Returns:
        List of chunk IDs for the added chunks

    Raises:
        ValueError: If metadatas length doesn't match texts length

    Example:
        >>> adapter = MemoryAdapter()
        >>> texts = [
        ...     "AI agents are autonomous",
        ...     "Machine learning powers AI",
        ...     "NLP enables text understanding"
        ... ]
        >>> metadatas = [
        ...     {"topic": "AI"},
        ...     {"topic": "ML"},
        ...     {"topic": "NLP"}
        ... ]
        >>> chunk_ids = adapter.add_texts(texts, metadatas)
    """
    if metadatas is not None and len(metadatas) != len(texts):
        raise ValueError(
            f"metadatas length ({len(metadatas)}) must match texts length ({len(texts)})"
        )

    chunk_ids = []
    for i, text in enumerate(texts):
        metadata = metadatas[i] if metadatas else None
        chunk_id = self.add_text(text, metadata)
        chunk_ids.append(chunk_id)

    return chunk_ids

add_document(document, metadata=None, chunk_size=500, chunk_overlap=50, separator='\n\n')

Add a document with automatic chunking.

The document will be split into chunks based on the specified parameters. Each chunk will inherit the document's metadata with an additional 'chunk_index' field.

Parameters:

Name Type Description Default
document str

Full document text

 required
metadata dict[str, Any] | None

Optional metadata for the document (inherited by all chunks)

 None
chunk_size int

Target size for each chunk (in characters)

 500
chunk_overlap int

Number of characters to overlap between chunks

 50
separator str

Separator to use for splitting (default: paragraph breaks)

 '\n\n'

Returns:

Type Description
list[str]

List of chunk IDs for the created chunks
Example

adapter = MemoryAdapter() long_text = "..." # Long document chunk_ids = adapter.add_document( ... long_text, ... metadata={"source": "paper.pdf", "author": "Alice"}, ... chunk_size=500, ... chunk_overlap=50 ... )

Source code in src/nlql/adapters/memory.py 
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
def add_document(
    self,
    document: str,
    metadata: dict[str, Any] | None = None,
    chunk_size: int = 500,
    chunk_overlap: int = 50,
    separator: str = "\n\n",
) -> list[str]:
    """Add a document with automatic chunking.

    The document will be split into chunks based on the specified parameters.
    Each chunk will inherit the document's metadata with an additional
    'chunk_index' field.

    Args:
        document: Full document text
        metadata: Optional metadata for the document (inherited by all chunks)
        chunk_size: Target size for each chunk (in characters)
        chunk_overlap: Number of characters to overlap between chunks
        separator: Separator to use for splitting (default: paragraph breaks)

    Returns:
        List of chunk IDs for the created chunks

    Example:
        >>> adapter = MemoryAdapter()
        >>> long_text = "..." # Long document
        >>> chunk_ids = adapter.add_document(
        ...     long_text,
        ...     metadata={"source": "paper.pdf", "author": "Alice"},
        ...     chunk_size=500,
        ...     chunk_overlap=50
        ... )
    """
    # Simple chunking strategy: split by separator first, then by size
    chunks = self._chunk_text(document, chunk_size, chunk_overlap, separator)

    chunk_ids = []
    base_metadata = metadata or {}

    for i, chunk_text in enumerate(chunks):
        chunk_metadata = base_metadata.copy()
        chunk_metadata["chunk_index"] = i
        chunk_metadata["total_chunks"] = len(chunks)

        chunk_id = self.add_chunk(chunk_text, chunk_metadata)
        chunk_ids.append(chunk_id)

    return chunk_ids

clear()

Clear all chunks from the adapter.

Example

adapter = MemoryAdapter() adapter.add_text("Some text") adapter.clear() len(adapter) == 0 True

Source code in src/nlql/adapters/memory.py 
277
278
279
280
281
282
283
284
285
286
287
def clear(self) -> None:
    """Clear all chunks from the adapter.

    Example:
        >>> adapter = MemoryAdapter()
        >>> adapter.add_text("Some text")
        >>> adapter.clear()
        >>> len(adapter) == 0
        True
    """
    self._chunks.clear()

__len__()

Get the number of chunks in the adapter.

Source code in src/nlql/adapters/memory.py 
289
290
291
def __len__(self) -> int:
    """Get the number of chunks in the adapter."""
    return len(self._chunks)

__repr__()

String representation of the adapter.

Source code in src/nlql/adapters/memory.py 
293
294
295
def __repr__(self) -> str:
    """String representation of the adapter."""
    return f"MemoryAdapter(chunks={len(self._chunks)})"