> ## Documentation Index
> Fetch the complete documentation index at: https://docs.artaios.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Knowledge Base Setup

> Configure and connect knowledge bases to your agents

Knowledge bases provide agents with access to domain-specific information, documents, and organizational knowledge through semantic search.

## Available Knowledge Base Types

<CardGroup cols={2}>
  <Card title="Milvus" icon="database">
    **Vector Database**

    Production-ready vector database with semantic search, hybrid search, and query expansion.
  </Card>

  <Card title="UKnow" icon="cloud">
    **Cloud Storage Search**

    Search documents in SharePoint, OneDrive, Google Drive, and Confluence via the UKnow API.
  </Card>
</CardGroup>

## Milvus Knowledge Base

Milvus is a vector database that enables semantic search over your documents using dense vector embeddings. Knowledge bases support customizable chunking strategies, embedding models, and structured data processing.

### Basic Configuration

```yaml theme={null}
agents:
  - name: "support_agent"
    agent_type: "llm_agent"
    
    knowledge_bases:
      - name: "company_docs"
        knowledge_base_type: "milvus"
        id: "kb_abc123"
        config:
          top_k: 10
          search_ef: 64
          strategy: "dense"
```

### Creating with API

Knowledge bases are created via the API with full control over processing configuration. See [Create Knowledge Base API](/api-reference/knowledge-bases/create) for complete documentation.

### Instance Fields

<ParamField path="name" type="string" required>
  Unique identifier for this knowledge base instance. Must contain only alphanumeric characters, hyphens, and underscores.
</ParamField>

<ParamField path="knowledge_base_type" type="string" required>
  Knowledge base type: `"milvus"` or `"uknow"`
</ParamField>

<ParamField path="enabled" type="boolean" default="true">
  Toggle the knowledge base on or off without removing it from configuration
</ParamField>

<ParamField path="id" type="string" optional>
  KnowledgeBase model ID for database lookup (required for Milvus to resolve collection info)
</ParamField>

<ParamField path="config" type="object" default="{}">
  Adapter-specific retrieval configuration (see below)
</ParamField>

<ParamField path="description" type="string" optional>
  Human-readable description of the knowledge base
</ParamField>

### Milvus Retrieval Configuration

The `config` object controls how documents are retrieved from Milvus:

<ParamField path="config.top_k" type="integer" default="10">
  Number of most relevant results to return

  **Range:** 1-1000
</ParamField>

<ParamField path="config.strategy" type="string" default="dense">
  Search strategy for retrieval

  **Options:**

  * `dense` - Vector-based semantic search only (default, fastest)
  * `hybrid` - Combines dense vector + BM25 keyword search
  * `bm25` - BM25 keyword-based search only
  * `hybrid_rrf` - Hybrid with Reciprocal Rank Fusion for improved ranking
</ParamField>

<ParamField path="config.search_ef" type="integer" optional>
  HNSW search parameter controlling accuracy vs speed trade-off

  **Higher values** = more accurate but slower
</ParamField>

<ParamField path="config.score_threshold" type="float" optional>
  Minimum similarity score threshold

  **Range:** 0.0-1.0\
  Only return results above this score
</ParamField>

<ParamField path="config.offset" type="integer" default="0">
  Number of results to skip (for pagination)
</ParamField>

<ParamField path="config.embedding_provider" type="string" default="mistral">
  Embedding provider for vectorization: `"mistral"`, `"azure_openai"`, or `"telekom_otc"`
</ParamField>

<ParamField path="config.embedding_model" type="string" default="mistral-embed">
  Embedding model name
</ParamField>

<ParamField path="config.query_model" type="string" optional>
  LLM model for query expansion. Must be one of the available LLM models.
</ParamField>

<ParamField path="config.max_num_query_expansions" type="integer" default="0">
  Number of expanded queries to generate (0-5). When greater than 0, uses the query\_model to generate variations of the original query for improved recall.
</ParamField>

<ParamField path="config.filter_expression" type="string" optional>
  Raw Milvus expression for pre-search filtering. Supports Milvus-native operators like `ARRAY_CONTAINS`.
</ParamField>

<ParamField path="config.filters" type="object" optional>
  Haystack-style metadata filters for document retrieval.
</ParamField>

## Usage Examples

### Basic Knowledge Base Agent

```yaml theme={null}
agents:
  - name: "support_agent"
    agent_type: "llm_agent"
    
    knowledge_bases:
      - name: "support_kb"
        knowledge_base_type: "milvus"
        id: "kb_support_001"
        config:
          top_k: 5
    
    system_prompt: |
      You are a customer support agent with access to our support documentation.
      
      Always search the knowledge base first for answers to customer questions.
      Provide accurate information based on our documentation.
```

### Hybrid Search with RRF Fusion

```yaml theme={null}
agents:
  - name: "research_agent"
    agent_type: "react_agent"
    
    tools:
      - "tavily_search"
    
    knowledge_bases:
      - name: "research_kb"
        knowledge_base_type: "milvus"
        id: "kb_research_001"
        config:
          strategy: "hybrid_rrf"  # Combines dense + BM25 with RRF
          top_k: 20
          search_ef: 128
          score_threshold: 0.7
    
    system_prompt: |
      You are a research assistant with access to internal research papers.
      
      Search the knowledge base for relevant research before using web search.
      Prioritize high-quality, relevant results.
```

### Query Expansion

```yaml theme={null}
agents:
  - name: "advanced_support_agent"
    agent_type: "llm_agent"
    
    knowledge_bases:
      - name: "product_kb"
        knowledge_base_type: "milvus"
        id: "kb_product_001"
        config:
          strategy: "dense"
          top_k: 15
          search_ef: 96
          
          # Query expansion for better recall
          query_model: "gpt-4o"
          max_num_query_expansions: 3
    
    system_prompt: |
      You are an expert product support agent.
      Use query expansion to find relevant documentation.
```

### UKnow Cloud Storage Search

```yaml theme={null}
agents:
  - name: "document_agent"
    agent_type: "llm_agent"
    
    knowledge_bases:
      - name: "sharepoint_docs"
        knowledge_base_type: "uknow"
        config:
          username: "user@company.com"
          drive_key: "SP"
          drive_ids: ["drive_abc123"]
          search_options:
            search_type: "similarity"
            fetch_k: 10
    
    system_prompt: |
      You have access to company SharePoint documents.
      Search the knowledge base to find relevant information.
```

### Multiple Knowledge Bases

```yaml theme={null}
agents:
  - name: "comprehensive_agent"
    agent_type: "llm_agent"
    
    knowledge_bases:
      - name: "technical_docs"
        knowledge_base_type: "milvus"
        id: "kb_tech_001"
        config:
          top_k: 10
      
      - name: "company_policies"
        knowledge_base_type: "milvus"
        id: "kb_policy_001"
        config:
          top_k: 5
    
    system_prompt: |
      You have access to multiple knowledge bases:
      - Technical documentation
      - Company policies
      
      Search the appropriate knowledge base based on the question type.
```

## Chunking Strategies

Knowledge bases support three chunking strategies configured during creation:

### Recursive (Default)

Splits text using multiple separators in order (paragraphs → sentences → words). Best for general documents.

**Parameters:**

* `strategy`: `"recursive"`
* `chunk_size`: Size in words/characters (default: 500)
* `chunk_overlap`: Overlap between chunks (default: 50)
* `split_by`: `"word"` | `"char"` (only these two options)
* `recursive_separators`: Array of separators to try in order

**Example:**

```json theme={null}
{
  "strategy": "recursive",
  "chunk_size": 500,
  "chunk_overlap": 50,
  "split_by": "word",
  "recursive_separators": ["\n\n", "\n", ". ", " "]
}
```

### Hierarchical

Creates multi-level chunks preserving document structure. Ideal for academic papers and structured documents.

**Parameters:**

* `strategy`: `"hierarchical"`
* `hierarchical_block_sizes`: Descending array of block sizes (e.g., `[700, 350, 150]`)
* `chunk_overlap`: Overlap between chunks
* `split_by`: `"word"` | `"sentence"`

**Example:**

```json theme={null}
{
  "strategy": "hierarchical",
  "chunk_overlap": 70,
  "split_by": "word",
  "hierarchical_block_sizes": [700, 350, 150]
}
```

### Fixed

Simple fixed-size chunks. Fastest processing for straightforward documents.

**Parameters:**

* `strategy`: `"fixed"`
* `chunk_size`: Fixed chunk size
* `chunk_overlap`: Overlap between chunks
* `split_by`: `"word"` | `"sentence"`

**Example:**

```json theme={null}
{
  "strategy": "fixed",
  "chunk_size": 500,
  "chunk_overlap": 50,
  "split_by": "word"
}
```

## Structured Data Processing

CSV and Excel files (`.csv`, `.xlsx`, `.xls`) support specialized processing:

### Configuration

```json theme={null}
{
  "document_config": {
    "strategy": "recursive",
    "chunk_size": 500,
    "chunk_overlap": 50,
    "split_by": "word"
  },
  "structured_config": {
    "rows_per_batch": 10,
    "table_format": "csv",
    "csv_content_column": "text",
    "csv_conversion_mode": "row"
  }
}
```

### Structured Config Parameters

<ParamField path="rows_per_batch" type="integer" default="10">
  Number of rows to combine into one searchable chunk (1-20)

  **Lower values** = More precise retrieval, slower ingestion\
  **Higher values** = Faster ingestion, broader context
</ParamField>

<ParamField path="table_format" type="string" default="csv">
  Output format for table data

  **Options:**

  * `csv` - Comma-separated values
  * `markdown` - Markdown table format
</ParamField>

<ParamField path="csv_content_column" type="string" default="text">
  Column name containing the main text content (CSV only)

  Required for `row` mode processing
</ParamField>

<ParamField path="csv_conversion_mode" type="string" default="row">
  Processing mode for CSV files

  **Options:**

  * `row` - One document per row (precise retrieval)
  * `file` - One document per file (holistic context)
</ParamField>

<ParamField path="use_streaming" type="boolean" default="false">
  If enabled, uses an openpyxl streaming mode for Excel processing to avoid loading entire sheets into memory. Recommended for Excel files larger than 50MB.
</ParamField>

## Embedding Providers

Choose your embedding model during knowledge base creation:

### Azure OpenAI

```json theme={null}
{
  "provider": "azure_openai",
  "model": "text-embedding-ada-002",
  "dimensions": 1536
}
```

**Available model:**

* `text-embedding-ada-002` (1536 dimensions)

### Mistral AI

```json theme={null}
{
  "provider": "mistral",
  "model": "mistral-embed",
  "dimensions": 1024
}
```

**Available model:**

* `mistral-embed` (1024 dimensions)

### Telekom OTC

```json theme={null}
{
  "provider": "telekom_otc",
  "model": "text-embedding-bge-m3",
  "dimensions": 1024
}
```

**Available models:**

* `text-embedding-bge-m3` (1024 dimensions) - BGE multilingual model
* `jina-embeddings-v2-base-de` (768 dimensions) - German-optimized
* `jina-embeddings-v2-base-code` (768 dimensions) - Code-optimized
* `tsi-embedding-colqwen2-2b-v1` (1024 dimensions) - TSI ColQwen2

## Best Practices

### Chunking

<Tip>
  **General Documents**: Use recursive strategy with 500 word chunks and 50 word overlap.
</Tip>

<Tip>
  **Code Files**: Use smaller chunks (300 words) with recursive separators `["\n\n", "\n", " "]`.
</Tip>

<Tip>
  **Research Papers**: Use hierarchical strategy with block sizes `[700, 350, 150]` to preserve structure.
</Tip>

<Tip>
  **CSV/Excel Data**: Start with `rows_per_batch: 10` and adjust based on row size and retrieval precision needs.
</Tip>

<Warning>
  Chunk overlap must be less than chunk size (or smallest block size for hierarchical). The API will reject invalid configurations.
</Warning>

### Retrieval

<Tip>
  **Top K Selection**: Start with `top_k: 5-10` for most use cases. Increase if you need more context.
</Tip>

<Tip>
  **Search Accuracy**: Use `search_ef: 64` for balanced performance. Increase to 128+ for higher accuracy needs.
</Tip>

<Tip>
  **Metric Type**: Use `COSINE` for most semantic search applications as it's normalized and works well with embeddings.
</Tip>

<Tip>
  **Score Threshold**: Set a `score_threshold` (e.g., 0.7) to filter out low-relevance results.
</Tip>

<Warning>
  Higher `search_ef` values improve accuracy but increase query latency. Balance based on your performance requirements.
</Warning>

## Next Steps

<CardGroup cols={2}>
  <Card title="Agent Capabilities" icon="wand-magic-sparkles" href="/documentation/agent_config/agent-capabilities">
    Explore all agent capabilities including tools and streaming
  </Card>

  <Card title="Agent Types" icon="robot" href="/documentation/agent_config/types_config/agent-types">
    Learn about different agent types
  </Card>
</CardGroup>