# Agent Orchestration

A production-ready Python-based multi-agent orchestration system for coordinating AI coding agents across CLI and API interfaces.

## Overview

This system orchestrates multiple AI coding agents, with interactive/web authentication as the primary mode for CLI agents and API keys reserved for specialized tasks.

**CLI Agents (Control Plane A):**
- Claude Code - Interactive coding with human oversight
- Gemini CLI - Large context analysis, search grounding
- Codex CLI - CI/CD automation, tiered autonomy

**API Agents (Control Plane B):**
- Claude Agent SDK - Programmatic Claude Code
- OpenAI Agents SDK - Handoffs, sessions, usage tracking
- Gemini API - Direct API calls

## Key Features

### Core Orchestration
- **Unified Adapter Layer** - Single interface for all agent types
- **Project Journal Protocol** - Context handoff between agents
- **Four-Tier Risk Gate** - LOW/MEDIUM/HIGH/CRITICAL autonomy levels
- **Health Monitoring** - Stuck agent detection, auto-prompting
- **Budget Controls** - Per-agent, per-MCP-server, per-tool limits
- **Merge Gate** - Protected branch controls with readiness checks
- **Secret Handling** - Log redaction, never-store patterns

### Agent Isolation & Recovery
- **Container/Tmux Isolation** - Docker, Podman, or tmux-based agent environments
- **Run-Until-Done Mode** - Automatic retry with configurable attempts
- **Credential Auto-Injection** - SSH keys, AWS credentials, Git config

### Multi-Agent Coordination
- **Voting/Consensus** - Multi-agent decision making with quorum types
- **JSON Workflows** - Declarative workflow definitions with dependencies
- **Swarm Intelligence** - Task decomposition and result aggregation
- **Agent Handoffs** - Context transfer between agents with state preservation
- **Shared Memory** - Thread-safe key-value storage for workflows

### Plugin Architecture
- **Adapter Registry** - Hot-reload and plugin discovery
- **Capability Registry** - Skill-based agent routing
- **Configurable Risk Policies** - JSON/YAML-based risk rules

### CLI State Integration
- **Native State File Reading** - Claude, Codex, Gemini state parsing
- **Subscription Tier Tracking** - Claude Max, ChatGPT Plus, Gemini Pro support
- **Interaction Detection** - Detect when CLI is waiting for input
- **Auto-Response Handler** - Automated approval handling with escalation

### Observability & UX
- **Terminal UI Dashboard** - Rich TUI with agent status and task queues
- **Session Tracing** - LangSmith-style span tracing with replay
- **Cost Optimization** - GitHub-style budget alerts and recommendations
- **Dashboard REST API** - Full API for monitoring and control

## Quick Start

### Prerequisites

- Python 3.11+
- tmux (for workspace isolation)
- Git (for worktrees)
- Claude Code CLI, Gemini CLI, Codex CLI (as needed)

### Installation

```bash
# Clone the repository
git clone <repo-url>
cd Agent_Orchestration

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -e .

# Copy environment template
cp .env.example .env
# Edit .env with your API keys
```

### Configuration

API keys are optional and intended for specialized tasks only. For day-to-day work, use interactive/web authentication with the CLI agents.

Edit `.env` if you need API access:

```bash
ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...
GOOGLE_API_KEY=AIza...
```

### Interactive Authentication (Primary)

Authenticate each CLI agent before running the orchestrator:

1. Launch each CLI once and complete the web/interactive login flow.
2. Confirm each CLI is operational with a simple prompt or status command.
3. Keep API keys unset unless a task explicitly requires API access.

See `ops/runbooks/cli-authentication.md` for operational guidance.

### Running the Orchestrator

```bash
# Start the orchestrator
python -m agent_orchestrator

# Or use the CLI script
python scripts/run_orchestrator.py

# Start the TUI dashboard
python -m agent_orchestrator.tui

# Start the REST API
python -m agent_orchestrator.api
```

## Architecture

```
┌─────────────────────────────────────────────────────────────────────┐
│                         User Interface                               │
│   CLI Commands  │  Terminal Dashboard (TUI)  │  REST API            │
├─────────────────────────────────────────────────────────────────────┤
│                      Python Orchestrator                             │
├─────────────────────────────────────────────────────────────────────┤
│  Task Router │ Risk Gate │ Budget │ Workflow Engine │ Swarm        │
├─────────────────────────────────────────────────────────────────────┤
│  Control Loop │ Health Check │ Voting │ Handoff Manager            │
├─────────────────────────────────────────────────────────────────────┤
│                       Adapter Layer                                  │
├─────────────────────────┬───────────────────────────────────────────┤
│   Control Plane A (CLI) │         Control Plane B (API)             │
│                         │                                            │
│  Claude Code            │  Claude Agent SDK                          │
│  Gemini CLI             │  OpenAI Agents SDK                         │
│  Codex CLI              │  Gemini API                                │
├─────────────────────────┴───────────────────────────────────────────┤
│  Memory Layer: Operational │ Knowledge │ Working │ Shared Memory    │
├─────────────────────────────────────────────────────────────────────┤
│  Persistence: SQLite │ Tracing Storage │ ops/ Files                 │
└─────────────────────────────────────────────────────────────────────┘
```

## Project Structure

```
Agent_Orchestration/
├── src/agent_orchestrator/
│   ├── adapters/          # Agent adapters (CLI + API)
│   ├── persistence/       # SQLite database layer
│   ├── journal/           # Project journal protocol
│   ├── control/           # Control loop, health checks
│   ├── risk/              # Risk classification, configurable policies
│   ├── interrupt/         # Human approval interfaces
│   ├── budget/            # Cost controls, alerts, optimization
│   ├── merge/             # Merge gate
│   ├── workspace/         # tmux + worktree management
│   ├── secrets/           # Secret redaction
│   ├── isolation/         # Container/tmux isolation
│   ├── voting/            # Multi-agent voting/consensus
│   ├── workflows/         # JSON workflow engine
│   ├── plugins/           # Plugin system, capabilities
│   ├── swarm/             # Swarm intelligence patterns
│   ├── cli_state/         # CLI state file readers
│   ├── subscriptions/     # Tier tracking and limits
│   ├── interactions/      # Interaction detection, auto-response
│   ├── api/               # Dashboard REST API
│   ├── tui/               # Terminal UI Dashboard
│   ├── tracing/           # Session tracing and replay
│   ├── coordination/      # Agent handoffs, shared memory
│   └── observability/     # Alerts, monitoring
├── tests/                 # Unit, integration, e2e tests (925+)
├── docs/                  # Architecture, operations docs
├── ops/                   # Runbooks, prompts, decisions
├── examples/              # Usage examples
├── scripts/               # Utility scripts
├── project_state.json     # Machine-readable state (generated)
└── agent_journal.md       # Human-readable log (generated)
```

## Documentation

### Core Documentation
- [ARCHITECTURE.md](docs/ARCHITECTURE.md) - System architecture
- [IMPLEMENTATION_PLAN.md](docs/IMPLEMENTATION_PLAN.md) - Phased implementation plan
- [IMPROVEMENTS.md](docs/IMPROVEMENTS.md) - Improvement roadmap and status
- [OPERATIONS.md](docs/OPERATIONS.md) - Runbooks and troubleshooting

### Phase Planning
- [PHASE_5_CLI_STATE_INTEGRATION.md](docs/PHASE_5_CLI_STATE_INTEGRATION.md) - CLI state integration
- [PHASE_6_ADVANCED_ORCHESTRATION.md](docs/PHASE_6_ADVANCED_ORCHESTRATION.md) - Advanced orchestration & UX
- [PHASE_7_DOCUMENTATION_REFINEMENT.md](docs/PHASE_7_DOCUMENTATION_REFINEMENT.md) - Documentation refinement

### Research & Recommendations
- [RESEARCH.md](docs/RESEARCH.md) - Technical research and design decisions
- [IMPROVEMENT_RECOMMENDATIONS.md](docs/IMPROVEMENT_RECOMMENDATIONS.md) - Detailed recommendations

## Risk Levels

| Level | Behavior | Examples |
|-------|----------|----------|
| LOW | Auto-allowed | Read files, run tests |
| MEDIUM | Edits OK, commands need approval | npm install, git push |
| HIGH | Suggest-only, ask user | Auth changes, migrations |
| CRITICAL | Auto-reject | Force push, terraform destroy |

## Testing

```bash
# Install dev dependencies
pip install -e ".[dev]"

# Run all tests (925+)
pytest

# Run specific test categories
pytest tests/unit/
pytest tests/integration/

# Run with coverage
pytest --cov=agent_orchestrator

# Run specific test file
pytest tests/unit/test_adapters.py -v
```

## Current Status

**Overall Readiness: 99% Production-Ready** (Phases 1-6 Complete)

| Category | Status | Notes |
|----------|--------|-------|
| Code Quality | 98% | Architecture, plugin system, swarm patterns |
| Error Handling | 90% | Exception hierarchy, retry mechanisms |
| Testing | 99% | 925+ passing, 0 failing |
| Documentation | 60% | Phase 7 in progress |
| Observability | 95% | Alerts, tracing, cost optimization |
| Extensibility | 98% | Plugins, capabilities, state readers |
| UX | 90% | TUI Dashboard, budget alerts |

## License

MIT License - See [LICENSE](./LICENSE) for details.

## Contributing

1. Read [IMPROVEMENTS.md](docs/IMPROVEMENTS.md) for current phase
2. Follow the coding standards in the existing codebase
3. Ensure all tests pass before submitting
4. Update documentation for any new features

## Acknowledgments

- [Claude Agent SDK](https://github.com/anthropics/claude-agent-sdk-python)
- [OpenAI Agents SDK](https://github.com/openai/openai-agents-python)
- [CrewAI](https://github.com/crewAIInc/crewAI) - Multi-agent patterns
- [Swarms](https://github.com/kyegomez/swarms) - Swarm intelligence
- [Claude Squad](https://github.com/smtg-ai/claude-squad) - TUI patterns
- [Agent Sessions](https://github.com/jazzyalex/agent-sessions) - State file reading
