> ## Documentation Index
> Fetch the complete documentation index at: https://docs.artaios.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge Bases API Overview

> Create and manage knowledge bases for agent retrieval

The Knowledge Bases API allows you to create and manage knowledge bases that agents can query for information. Knowledge bases organize collections and data sources for AI-powered retrieval with customizable chunking strategies and embedding models.

## Key Features

* **Customizable Chunking**: Recursive, hierarchical, or fixed strategies
* **Structured Data Support**: Process CSV and Excel files
* **Multiple Embedding Providers**: Azure OpenAI, Mistral, Telekom OTC
* **AI-Powered Retrieval**: Vector-based semantic search
* **Milvus Processing**: Production-ready vector database engine
* **Flexible Data Selection**: Combine collections and individual data sources
* **Status Monitoring**: Track processing progress with detailed metadata
* **Agent Integration**: Direct use in agent configurations

## Authentication

All endpoints require authentication using your API key:

* **API Key**: `x-api-key: <key>`

## Available Endpoints

<CardGroup cols={2}>
  <Card title="List Knowledge Bases" icon="list" href="/api-reference/knowledge-bases/list">
    Retrieve all knowledge bases
  </Card>

  <Card title="Create Knowledge Base" icon="plus" href="/api-reference/knowledge-bases/create">
    Create a new knowledge base
  </Card>

  <Card title="Get Knowledge Base" icon="eye" href="/api-reference/knowledge-bases/get">
    Retrieve knowledge base details
  </Card>

  <Card title="Update Knowledge Base" icon="pen" href="/api-reference/knowledge-bases/update">
    Modify knowledge base settings
  </Card>

  <Card title="Delete Knowledge Base" icon="trash" href="/api-reference/knowledge-bases/delete">
    Remove a knowledge base
  </Card>

  <Card title="Check Status" icon="activity" href="/api-reference/knowledge-bases/check-status">
    Monitor processing status
  </Card>

  <Card title="Available Collections" icon="folder" href="/api-reference/knowledge-bases/available-collections">
    List collections ready for use
  </Card>
</CardGroup>

## Processing Engine

**Milvus Vector Database**

* High-performance vector similarity search
* Scalable for large document collections
* Advanced HNSW indexing algorithms
* Real-time updates and queries
* Production-ready for enterprise deployments

<Info>
  Milvus is currently the only supported processing engine. Additional engines may be added in future releases.
</Info>

## Knowledge Base Lifecycle

### 1. Creation

Define knowledge base with data sources, chunking strategy, and embedding configuration:

```json theme={null}
{
  "name": "Product Knowledge Base",
  "description": "Product documentation and guides",
  "processing_engine": "milvus",
  "selected_collections": ["coll_123"],
  "selected_datasources": ["ds_456"],
  "chunking_strategy": {
    "strategy": "recursive",
    "chunk_size": 500,
    "chunk_overlap": 50,
    "split_by": "word",
    "recursive_separators": ["\n\n", "\n", ". ", " "]
  },
  "embedding_config": {
    "provider": "mistral",
    "model": "mistral-embed",
    "dimensions": 1024
  }
}
```

### 2. Processing

Knowledge base processes data source content in two phases:

**Phase 1: Chunking (0-20% progress)**

* Extract text from documents (PDF, DOCX, TXT, MD, LOG)
* Process structured data (CSV, Excel) with custom row batching
* Apply chunking strategy to split into searchable chunks
* Track per-document progress

**Phase 2: Embedding & Indexing (20-100% progress)**

* Generate embeddings via batch API (Mistral, Azure OpenAI, or Telekom OTC)
* Stream embeddings to Milvus vector database
* Build HNSW indices for fast similarity search
* Validate data quality and store metadata

### 3. Completion

Ready knowledge bases can be used by agents for retrieval. Check `status`, `progress`, and `success_rate` to monitor health.

## Knowledge Base Status

<AccordionGroup>
  <Accordion title="pending">
    Knowledge base created but processing not started
  </Accordion>

  <Accordion title="running">
    Collections are being processed and indexed
  </Accordion>

  <Accordion title="completed">
    All collections processed and ready for queries
  </Accordion>

  <Accordion title="failed">
    Processing failed - check error details and logs
  </Accordion>
</AccordionGroup>

## Response Schema

Knowledge base responses include comprehensive processing details:

<ResponseField name="datasources" type="array">
  Array of data source objects included in this knowledge base
</ResponseField>

<ResponseField name="datasource_count" type="integer">
  Total number of data sources
</ResponseField>

<ResponseField name="ingested_datasources" type="array">
  Successfully processed data sources with full metadata
</ResponseField>

<ResponseField name="failed_datasources" type="array">
  Data sources that failed processing with error details
</ResponseField>

<ResponseField name="datasource_errors" type="object">
  Map of datasource\_id to error messages for failed sources
</ResponseField>

<ResponseField name="datasource_snapshots" type="object">
  Snapshot of datasource details at ingestion time with ingestion status
</ResponseField>

<ResponseField name="success_rate" type="number">
  Percentage of successfully processed data sources (0-100)
</ResponseField>

<ResponseField name="chunking_strategy" type="object">
  Applied chunking configuration (recursive, hierarchical, or fixed)
</ResponseField>

<ResponseField name="embedding_config" type="object">
  Embedding model configuration (provider, model, dimensions)
</ResponseField>

<ResponseField name="processing_metadata" type="object">
  Processing progress details including current phase and document being processed
</ResponseField>

## Supported File Types

**Documents:**

* PDF, DOC, DOCX
* TXT, MD (Markdown), LOG

**Structured Data:**

* CSV (row-level or file-level processing)
* XLSX, XLS (Excel with batch row processing)

## Chunking Strategies

Three strategies available for document processing:

### Recursive

Splits text using multiple separators in order. Best for general documents.

* Configurable separators (paragraphs, sentences, words)
* Flexible chunk sizes and overlap
* Supports word or character splitting only

### Hierarchical

Creates multi-level chunks preserving document structure. Ideal for research papers.

* Multiple block sizes in descending order
* Maintains document hierarchy
* Word splitting only

### Fixed

Simple fixed-size chunks. Fastest processing.

* Consistent chunk sizes
* Configurable overlap
* Word splitting only

See [Create Knowledge Base](/api-reference/knowledge-bases/create#chunking-strategies) for detailed configuration.

## Embedding Providers

**Azure OpenAI**

* text-embedding-ada-002 (1536 dimensions)

**Mistral AI**

* mistral-embed (1024 dimensions)

**Telekom OTC**

* text-embedding-bge-m3 (1024 dimensions)
* jina-embeddings-v2-base-de (768 dimensions) - German-optimized
* jina-embeddings-v2-base-code (768 dimensions) - Code-optimized
* tsi-embedding-colqwen2-2b-v1 (1024 dimensions)

## Agent Integration

Use knowledge bases in agent configurations:

```yaml theme={null}
name: "Support Agent"
agent_type: "llm_agent"
system_prompt: "You are a helpful support agent..."
knowledge_bases:
  - name: "product_kb"
    knowledge_base_type: "milvus"
    knowledge_base_id: "kb_123"
    retrieval_config:
      strategy: "dense"
      top_k: 10
      search_ef: 64
      metric_type: "COSINE"
```

### Retrieval Configuration

* **Search parameters**: Control relevance and results
* **Context integration**: How retrieved content is used
* **Fallback behavior**: When no relevant content is found

<Tip>
  Test knowledge base queries before deploying agents to ensure relevant results.
</Tip>

## Performance Considerations

### Indexing Time

* **Collection size**: Larger collections take longer to process
* **Content complexity**: Rich documents require more processing
* **Engine choice**: Different engines have varying performance characteristics

### Query Performance

* **Index optimization**: Proper indexing improves search speed
* **Result filtering**: Limit results for faster responses
* **Caching**: Frequently accessed content is cached

### Scalability

* **Concurrent queries**: Multiple agents can query simultaneously
* **Update frequency**: How often content changes
* **Storage requirements**: Vector storage grows with content

<Warning>
  Large knowledge bases may require significant processing time. Plan for appropriate indexing windows.
</Warning>

## Monitoring and Troubleshooting

### Health Checks

* **Processing progress**: Track completion percentage
* **Error rates**: Monitor failed data sources
* **Query performance**: Response times and accuracy

### Common Issues

* **Collection dependencies**: Ensure collections are completed
* **Processing failures**: Check data source formats and content
* **Memory limits**: Large knowledge bases may hit resource limits

<Info>
  Use the status endpoint to monitor knowledge base health and processing progress.
</Info>