Testing & Verification

This document describes the verified SDK functionality with real infrastructure.

End-to-End Test Results

All execution modes have been tested against real infrastructure (isA_Model, isA_MCP, NATS, Redis, PostgreSQL):

Test	Status	Duration	Details
Reactive Mode (Basic)	✅ PASS	~8s	Simple query execution
Reactive Mode (Tools)	✅ PASS	~7s	Tool discovery & availability
Collaborative Mode	✅ PASS	~2.5s	SessionServiceCheckpointer
Proactive Triggers	✅ PASS	~100ms	NATS JetStream triggers
HIL Functions	✅ PASS	<1ms	Interrupt tracking
Skills System	✅ PASS	<1ms	6 built-in skills
Streaming + Skills	✅ PASS	~3-4s	Skills + streaming
Session Resumption	✅ PASS	~1.5s	Checkpoint restore
Direct Tool Execution	✅ PASS	~40ms	MCP tool execution
Tool Discovery	✅ PASS	<10ms	Semantic tool search
A2A Server + Client (Local)	✅ PASS	~60-120s	8101 → 8102 JSON-RPC call via `/a2a`

A2A Verification

A2A was validated with:

auth_service: http://localhost:8201
agent A: http://localhost:8101
agent B: http://localhost:8102

Validated endpoints:

GET /.well-known/agent-card.json
POST /a2a

Validated flow:

POST /api/v1/a2a/test-client on 8101
target: http://localhost:8102/a2a
result: success with JSON-RPC response

See full setup and code examples: A2A guide

Verified Execution Modes

Reactive Mode

Standard request-response interaction:


from isa_agent_sdk import query, ISAAgentOptions
 
options = ISAAgentOptions(
    model="gpt-4o-mini",
    session_id="test_session",
    max_iterations=10
)
 
async for msg in query("What is 2 + 2? Answer in one word.", options=options):
    if msg.is_text:
        print(msg.content, end="")

Real Test Output:


Four

Verified:

Messages stream correctly (8 total messages)
Message types: session_start, system, text, node_exit, result, session_end
Session lifecycle managed properly

Collaborative Mode (Checkpointing)

Durable execution with state persistence:


from isa_agent_sdk import query, ISAAgentOptions, ExecutionMode
 
options = ISAAgentOptions(
    execution_mode=ExecutionMode.COLLABORATIVE,
    session_id="my_durable_task",
    checkpoint_frequency=2
)
 
async for msg in query("Count from 1 to 5", options=options):
    print(msg.content, end="" if msg.is_text else "\n")

Real Test Output:


Checkpointer type: SessionServiceCheckpointer
Backend: session_service
Total messages: 10
Session ID: test_collab_1769511181

Verified:

SessionServiceCheckpointer correctly saves state
Session can be resumed after interruption
Checkpoint ID properly tracked

Proactive Mode (Event Triggers)

Event-driven agent activation:


from isa_agent_sdk import (
    initialize_triggers,
    register_trigger,
    get_user_triggers,
    TriggerType
)
 
await initialize_triggers()
 
trigger_id = await register_trigger(
    user_id="user-123",
    trigger_type=TriggerType.THRESHOLD,
    description="Price alert",
    conditions={"threshold_value": 5.0, "direction": "down"},
    action_config={"prompt": "Analyze the drop"}
)
 
triggers = await get_user_triggers("user-123")

Verified:

NATS JetStream connects and initializes
Triggers register and unregister correctly
Trigger statistics tracked

Interactive Mode (Human-in-the-Loop)

Approval workflows:


from isa_agent_sdk import get_hil, get_hil_stats, clear_hil_history
 
hil = get_hil()
stats = get_hil_stats()
print(f"Total interrupts: {stats.total}")

Verified:

HIL service available
Interrupt statistics tracked
History management works

Skills System

Specialized behaviors via prompt injection:


from isa_agent_sdk import (
    list_builtin_skills,
    load_skill,
    activate_skills
)
 
skills = list_builtin_skills()
# ['code-review', 'debug', 'refactor', 'test-writer', 'documentation', 'security-audit']
 
skill = await load_skill("code-review")
print(f"Skill: {skill.name} - {skill.description}")
 
injection = activate_skills("code-review", "debug")

Real Test Output:


Built-in skills: ['code-review', 'debug', 'refactor', 'test-writer', 'documentation', 'security-audit']
Loaded skill: {
  name: 'code-review',
  description: 'Expert code reviewer focusing on quality, security...',
  triggers: ['review', 'code review', 'check this code', 'review my code']
}
Injection length: 847 chars

Verified:

6 built-in skills available
Skills load correctly with metadata
Activation returns prompt injection

Direct Tool Execution

Execute tools without agent:


from isa_agent_sdk import execute_tool
 
result = await execute_tool(
    tool_name="get_current_time",
    tool_args={},
    session_id="direct_exec"
)
print(result.tool_result_value)

Real Test Output:


result.type = 'tool_result'
result.tool_result_value = {
  'result': {
    'content': [{
      'type': 'text',
      'text': '{"iso": "2026-01-27T10:41:53.751918+00:00", "date": "2026-01-27", ...}'
    }]
  }
}

Verified:

MCP tools execute correctly (40ms average)
Results return with proper typing
Error handling works

Tool Discovery

Find tools semantically:


from isa_agent_sdk import get_available_tools
 
# Get all tools
all_tools = await get_available_tools()
print(f"Total tools: {len(all_tools)}")
 
# Semantic search
web_tools = await get_available_tools(
    user_query="search the web",
    max_results=5
)

Real Test Output:


Total tools: 20
Sample tools: ['get_current_time', 'get_current_date', 'get_authorization_requests',
               'approve_authorization', 'get_monitoring_metrics']

Verified:

Tool discovery from MCP works (20 tools available)
Semantic search available

Infrastructure Requirements

The tests require:

Service	Port	Purpose
isA_Model	8082	LLM inference
isA_MCP	8081	Tool/prompt/resource access
NATS	4222	Event triggers
Redis	6379	Tool profiling, RL data
PostgreSQL	5432	Traces, sessions

Running Tests


# Set environment
export ISA_MODEL_URL=http://localhost:8082
export ISA_MCP_URL=http://localhost:8081
 
# Run all tests
python tests/test_e2e_execution_modes.py

Known Limitations

Tool Calling in ReasonNode: The ReasonNode uses DeepSeek-R1 for reasoning, which doesn’t support OpenAI-style tool calling. Direct tool execution via execute_tool() works correctly.
NATS Consumer Naming: Some NATS consumer names with special characters log warnings but don’t affect functionality.
Semantic Tool Search: Empty results for some queries if tools aren’t indexed in MCP.

Next Steps

Quickstart - Get started
Options - Configuration reference
Deployment Guide - Production deployment