Aıon

Documentation

Chat API Reference

Agentic chat sessions with image and document context, streaming responses

Agentic Chat

Create chat sessions to have AI-powered conversations about your images and documents. The system uses specialized agents for search, analysis, document search, analytics, cross-referencing, and folder organization.

Tip: Save Tokens with Standalone Search

If you only need to search images or documents without multi-turn conversation, use the standalone search agents directly. They consume significantly fewer tokens by skipping session overhead and conversational context.

Session Management

POST/api/v2/chat/sessions

Create a new chat session

Request

{
  "title": "Equipment Analysis",       // optional, max 255 chars
  "initial_image_ids": ["550e8400-e29b-41d4-a716-446655440000", "660f9500-f39c-52e5-b827-557766550111"],  // optional, up to 1000 if use_all_images is false
  "use_all_images": true                // default: true
}

Response

{
  "session_id": "550e8400-e29b-41d4-a716-446655440000",
  "title": "Equipment Analysis",
  "created_at": "2025-01-15T10:30:00Z"
}
GET/api/v2/chat/sessions

List user's chat sessions with optional filtering

Request

// Query parameters:
?search=equipment    // Search in session titles and message content (max 200 chars)
&is_active=true      // Filter by active status (true/false)
&min_messages=1      // Filter sessions with at least N messages
&limit=20            // 1-100, default 20
&offset=0            // default 0

Response

{
  "sessions": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "title": "Equipment Analysis",
      "total_messages": 12,
      "total_tokens": 5400,
      "remaining_tokens": 94600,
      "remaining_messages": 88,
      "is_active": true,
      "use_all_images": true,
      "selected_image_count": 0,
      "last_message_at": "2025-01-15T11:00:00Z",
      "last_message_preview": "Based on the analysis...",
      "last_user_message": "What damage do you see?",
      "created_at": "2025-01-15T10:30:00Z",
      "updated_at": "2025-01-15T11:00:00Z"
    }
  ],
  "total_count": 5,
  "has_more": false
}
GET/api/v2/chat/sessions/{session_id}

Get session details with message history

Request

// Query parameters:
?include_messages=true   // Include messages in response (default: true)
&message_limit=50        // Limit number of messages returned (1-1000, optional)

Response

{
  "session": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "title": "Equipment Analysis",
    "total_messages": 4,
    "total_tokens": 1200,
    "remaining_tokens": 94000,
    "remaining_messages": 96,
    "is_active": true,
    "use_all_images": false,
    "selected_image_count": 2,
    "last_message_at": "2025-01-15T10:31:02Z",
    "last_message_preview": "I can see corrosion on...",
    "last_user_message": "What damage do you see?",
    "created_at": "2025-01-15T10:30:00Z",
    "updated_at": "2025-01-15T10:31:02Z"
  },
  "messages": [
    {
      "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "role": "user",
      "content": "What damage do you see?",
      "token_count": 12,
      "provider": null,
      "model": null,
      "processing_time_ms": null,
      "created_at": "2025-01-15T10:31:00Z",
      "image_context": null,
      "metadata": null
    },
    {
      "id": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
      "role": "assistant",
      "content": "I can see corrosion on...",
      "token_count": 450,
      "provider": "anthropic",
      "model": "claude-sonnet-4-20250514",
      "processing_time_ms": 1250,
      "created_at": "2025-01-15T10:31:02Z",
      "image_context": [
        {
          "image_id": "550e8400-e29b-41d4-a716-446655440000",
          "thumbnail_url": "https://...",
          "description": "Steel beam with rust",
          "confidence": 0.95,
          "title": "Beam photo",
          "stored_url": "https://...",
          "filename": "beam_001.jpg"
        }
      ],
      "metadata": {
        "vision_used": true,
        "offers_detailed_analysis": false
      }
    }
  ],
  "selected_image_ids": ["550e8400-e29b-41d4-a716-446655440000", "660f9500-f39c-52e5-b827-557766550111"],
  "context_summary": "2 images selected"
}
PATCH/api/v2/chat/sessions/{session_id}/title

Rename a chat session

Request

{
  "title": "Updated Session Title"  // required, 1-255 chars, trimmed
}

Response

{
  "session_id": "550e8400-e29b-41d4-a716-446655440000",
  "title": "Updated Session Title",
  "updated_at": "2025-01-15T11:00:00Z"
}
DELETE/api/v2/chat/sessions/{session_id}

Close an active chat session and release resources

Response

{
  "session_id": "550e8400-e29b-41d4-a716-446655440000",
  "closed_at": "2025-01-15T11:00:00Z",
  "total_messages": 12,
  "total_tokens": 5400
}

Messaging

POST/api/v2/chat/sessions/{session_id}/messages

Send message and get complete response (non-streaming)

Request

{
  "message": "What safety issues do you see in these images?",  // required, 1-10000 chars
  "force_detailed_analysis": false,  // optional, default false - force high-resolution image analysis
  "expected_image_ids": ["img_001", "img_002"]  // optional - validate/sync image selection
}

Response

{
  "message_id": "550e8400-e29b-41d4-a716-446655440000",
  "content": "Based on my analysis of the images...",
  "token_count": 450,
  "provider": "anthropic",
  "model": "claude-sonnet-4-20250514",
  "processing_time_ms": 2500,
  "remaining_tokens": 94550,
  "remaining_messages": 97
}
POST/api/v2/chat/sessions/{session_id}/messages/stream

Send message with streaming response using Server-Sent Events (SSE)

Request

{
  "message": "What are the key safety requirements?",  // required, 1-10000 chars
  "force_detailed_analysis": false,  // optional, default false
  "expected_image_ids": ["img_001", "img_002"]  // optional - validate/sync image selection
}

Response

// Server-Sent Events stream:


// connection - Initial connection confirmation
event: connection
data: {"type": "connection", "message_id": "msg_001", "status": "connected"}


// token - Individual response tokens
event: token
id: 1
data: {"type": "token", "content": "The", "accumulated": "The", "token_index": 0}


// status - Real-time processing status updates
event: status
data: {"type": "status", "phase": "searching", "message": "Searching documents...", "agent_name": "SearchAgent"}


// thinking - AI reasoning phase indicator
event: thinking
data: {"type": "thinking", "status": "processing"}


// thinking_step - Detailed agent reasoning with content
event: thinking_step
data: {"type": "thinking_step", "phase": "analysis", "agent_name": "AnalysisAgent", "display_name": "Analysis", "message": "Examining images...", "reasoning": "Examining image for..."}


// tool_invocation - Agent tool execution notification
event: tool_invocation
data: {"type": "tool_invocation", "tool_name": "search_images", "agent_name": "SearchAgent"}


// image_results - Images found during response
event: image_results
data: {"type": "image_results", "images": [...], "count": 5}


// complete - Response finished with full content and metadata
event: complete
data: {"type": "complete", "content": "...", "resolved_content": "...", "message_id": "550e8400-...", "session_id": "660f9500-...", "agents_used": ["SearchAgent", "AnalysisAgent"], "reasoning": "...", "has_results": true, "result_count": 5, "image_context": [...], "tool_document_data": [...], "tool_chunk_data": [...], "tool_link_data": [...], "result_refs": {...}}


// Optional fields in complete event:
// "scene_context": [...] - present when video scene results are included
// "folder_modified": true - present when a folder was modified (for cache invalidation)
// "pending_actions": [...] - present when the response requires user confirmation


// auth_error - Authorization failed during stream
event: auth_error
data: {"type": "auth_error", "error": "Session not found", "message": "Authorization failed during stream initialization"}


// error - Error occurred during processing
event: error
data: {"type": "error", "message": "You've reached your monthly chat message limit (200/200).", "code": "USAGE_LIMIT_EXCEEDED", "context": {"limit_type": "monthly_chat_messages", "current_usage": 200, "monthly_limit": 200, "reset_date": "2025-02-01", "upgrade_url": "/settings/billing"}}


// ping - Heartbeat (keep-alive during long operations)
event: ping
data: {"type": "ping"}


// close - Stream terminated
event: close
data: {"type": "close", "message_id": "msg_001"}

resolved_content in Complete Event

The complete event includes a resolved_content field — the response text with all markup patterns ([[ref:KEY]], [[img:ID|filename]], [[doc:ID|filename|p:N]], [[link:ID|title]]) replaced with human-readable text. Use this for CLI output or any context where you don't need interactive reference rendering. The original content field with markup is always included alongside for custom UI rendering.

auth_error Event

The auth_error event is emitted as the very first event if session authorization fails during stream initialization (e.g., session deleted or ownership changed between the pre-stream check and the actual stream start). When received, the stream terminates immediately. Your client should handle this by showing an error and redirecting the user to create a new session.

Context Management

PATCH/api/v2/chat/sessions/{session_id}/mode

Toggle between all images and selected images mode

Request

{
  "use_all_images": true  // required
}

Response

{
  "session_id": "550e8400-e29b-41d4-a716-446655440000",
  "use_all_images": true,
  "selected_image_count": 0,
  "updated_at": "2025-01-15T11:00:00Z"
}
PUT/api/v2/chat/sessions/{session_id}/images

Update which images are available in chat context (max 1000)

Request

{
  "image_ids": [
    "550e8400-e29b-41d4-a716-446655440000",
    "660f9500-f39c-52e5-b827-557766550111"
  ]
}

Response

{
  "session_id": "550e8400-e29b-41d4-a716-446655440000",
  "selected_image_ids": [
    "550e8400-e29b-41d4-a716-446655440000",
    "660f9500-f39c-52e5-b827-557766550111"
  ],
  "context_summary": "2 images selected",
  "estimated_tokens": 1500
}
PUT/api/v2/chat/sessions/{session_id}/documents

Update which documents are available in chat context (max 100)

Request

{
  "document_ids": [
    "550e8400-e29b-41d4-a716-446655440000",
    "660f9500-f39c-52e5-b827-557766550111"
  ]
}

Response

{
  "session_id": "550e8400-e29b-41d4-a716-446655440000",
  "selected_document_ids": [
    "550e8400-e29b-41d4-a716-446655440000",
    "660f9500-f39c-52e5-b827-557766550111"
  ],
  "document_count": 2
}
GET/api/v2/chat/images/all

Get all images available for chat context

Request

// Query parameters:
?limit=5000  // 1-5000, default 5000
&offset=0    // default 0

Response

{
  "image_ids": [
    "550e8400-e29b-41d4-a716-446655440000",
    "660f9500-f39c-52e5-b827-557766550111",
    "770a0600-a40d-63f6-c938-668877660222"
  ],
  "total_count": 150,
  "has_more": false
}
POST/api/v2/chat/images/search

Search for images using natural language queries

Request

{
  "query": "red cars from last week",  // required, 1-500 chars
  "limit": 50,                         // optional, 1-100, default 50
  "offset": 0                          // optional, default 0
}

Response

{
  "results": [
    {
      "image_id": "550e8400-e29b-41d4-a716-446655440000",
      "description": "Red sedan in parking lot",
      "confidence": 0.92,
      "relevance_score": 0.87,
      "thumbnail_url": "https://...",
      "medium_url": "https://...",
      "full_url": "https://...",
      "created_at": "2025-01-10T08:00:00Z"
    }
  ],
  "total_count": 15,
  "query_interpretation": {
    "original_query": "red cars from last week",
    "filters_applied": ["color: red", "date: last 7 days"],
    "search_type": "semantic"
  }
}

Image Search

The search engine understands natural language including context, synonyms, and filters. Example queries: "red cars from last week", "images with dogs and cats", "high confidence detections of people".

Plan Approval

POST/api/v2/chat/sessions/{session_id}/plans/{plan_id}/action

Approve or cancel an execution plan that requires user confirmation

Request

{
  "action": "approve"  // required: "approve" or "cancel"
}

Response

// --- Fast path (200): Plan completes within 2 seconds ---
{
  "success": true,
  "plan_id": "plan_abc123",
  "message": "Plan completed",
  "action_taken": "approve",
  "results": {
    "fast_path": true
  }
}


// --- Slow path (202 Accepted): Plan deferred to background ---
// Progress is streamed via WebSocket events (PlanExecutionStarted,
// PlanCompleted, PlanFailed).
{
  "success": true,
  "plan_id": "plan_abc123",
  "message": "Plan execution started. Progress will be streamed via WebSocket.",
  "action_taken": "approve"
}


// --- Cancelled plan (200) ---
{
  "success": true,
  "plan_id": "plan_abc123",
  "message": "Plan cancelled.",
  "action_taken": "cancel"
}

Dual Response Paths

If the plan completes quickly, you get a synchronous 200 response with results. For longer-running plans, you receive a 202 Accepted immediately and execution continues in the background. Subscribe to WebSocket events for real-time progress updates when you receive a 202.

Plan Approval Flow

When a complex request requires multiple agent operations, the streaming response may emit a plan_pending_approval event. Your application should:

  1. Detect the plan_pending_approval event in the stream
  2. Display the plan details to the user for review
  3. Call this endpoint with action: "approve" or "cancel"
  4. If you receive 200, the plan completed synchronously -- read the results
  5. If you receive 202, listen for WebSocket events for progress and completion

Error Responses

Additional error codes for plan approval:

  • 404 -- Plan not found or expired
  • 409 -- Plan is already being processed
  • 503 -- Service temporarily unavailable

Export

GET/api/v2/chat/sessions/{session_id}/export

Export chat session in various formats

Request

// Query parameters:
?format=markdown         // json | markdown | text (default: markdown)
&include_metadata=false  // Include metadata in export (default: false)

Response

// Returns file download with Content-Disposition header


// markdown format:
# Chat Session: Equipment Analysis


## User
What are the safety requirements?


## Assistant
Based on the documents, the key safety requirements are...


// json format:
{
  "session_id": "550e8400-e29b-41d4-a716-446655440000",
  "title": "Equipment Analysis",
  "messages": [...],
  "metadata": {...}
}

Related

Agentic Chat Guide

Step-by-step tutorial

Agents Reference

Direct AI-powered search

Documents Reference

Document management and search