docs · embedding
Embed the engine#
The hekma command is a thin layer over a Rust library, hekma-engine, and your own program can use the same library. In Hekma 0.11.0, embedding is the way to get supervision with no gaps, and the only way to use send and ACP prompt turns, because your program holds the engine open for as long as it runs. This page covers adding the crate, a complete quickstart, the API surface, and the errors and upgrades you need to plan for.
Who this is for#
Embed the engine when the CLI's command-scoped supervision is not enough:
- You want an agent watched around the clock, with crashes restarted and budgets enforced when they are crossed.
- You want to send an agent input with
send_input, or drive an ACP agent's prompt turns. In 0.11.0 the CLI cannot do either usefully, because a detached agent has no stdin and an attached one stops when its command exits. - You run engine-observed agents, whose metering proxy lives inside the process that started them.
- You are building a dashboard, a scheduler or a hosting product, and want events instead of polling
hekma agent list --json.
If you only need to start agents and check on them now and then, the CLI with --detach and a long-running hekma agent logs NAME --follow may be enough; see Supervision and detached runs.
Add the dependency#
[dependencies]
hekma-engine = "0.7"hekma-engine 0.7.0 is the engine inside Hekma 0.11.0. It needs Rust 1.96.1 or newer and is licensed Apache-2.0. It brings its own SQLite (compiled from source, so the first build takes a while), its own TLS stack for the metering proxy, and its own tokio runtime. You do not need tokio yourself unless you want to call the async methods.
The engine is versioned separately from the CLI. Within 0.x a minor version may break the API (see upgrading), so the "0.7" requirement above keeps you on 0.7.x until you choose to move.
Quickstart#
This program opens an engine on a scratch directory, registers an agent from an adapter manifest, sets a budget, subscribes to events, starts the agent, reads the fleet, and stops it. It is a trimmed copy of embedding-quickstart.rs, which uses its own binary as a stand-in agent; point exec at your agent instead.
use std::time::Duration;
use hekma_engine::broadcast::error::TryRecvError;
use hekma_engine::{AdapterRef, ConfigLayer, Engine};
fn main() -> Result<(), Box<dyn std::error::Error>> {
// 1. Open an engine on a state directory (None = the OS default).
let root = std::env::temp_dir().join("hekma-quickstart");
std::fs::create_dir_all(&root)?;
let engine = Engine::open(Some(root.clone()))?;
let host = engine.blocking();
// 2. Register an agent from an adapter.toml directory.
let manifest = root.join("hello-adapter");
std::fs::create_dir_all(&manifest)?;
std::fs::write(
manifest.join("adapter.toml"),
r#"contract_version = "1.0.0"
[adapter]
kind = "quickstart"
[lifecycle.start]
exec = "/path/to/your/agent"
[capabilities.pause]
linux = "guaranteed"
macos = "guaranteed"
windows = "best-effort"
[metering]
source = "self-reported"
"#,
)?;
host.register_with_adapter("hello-agent", &AdapterRef::Manifest(manifest))?;
// 3. Set a budget and read it back with its source layer.
host.set_config("hello-agent", "budget.tokens.cumulative", "100000")?;
let effective = host.effective_config("hello-agent", ConfigLayer::empty())?;
assert_eq!(effective.source_label("budget.tokens.cumulative"), Some("instance"));
// 4. Subscribe before starting, so the start's events arrive.
let mut events = host.subscribe();
// 5. Start: returns once the agent is running.
host.start("hello-agent")?;
// 6. Drain what has arrived, then read the fleet row.
loop {
match events.try_recv() {
Ok(event) => println!("event: {event:?}"),
Err(TryRecvError::Lagged(n)) => println!("missed {n} events"),
Err(TryRecvError::Empty | TryRecvError::Closed) => break,
}
}
let entry = host.fleet_entry("hello-agent")?;
println!("fleet: {} [{}] state={}", entry.name.as_str(), entry.kind, entry.state.as_str());
// 7. Stop, with a 5-second graceful window before the forced kill.
host.stop("hello-agent", Some(Duration::from_secs(5)))?;
Ok(())
}Run the real example from a clone of the repository:
git clone https://github.com/Ktesio/hekma.git
cd hekma
cargo run -p hekma-engine --example embedding-quickstartevent: Transition(TransitionEvent { schema_version: 1, instance: "hello-agent", prior_state: Registered, new_state: Starting, cause: Command { command: "start" }, at: "2026-10-22T19:37:36Z" })
event: Transition(TransitionEvent { schema_version: 1, instance: "hello-agent", prior_state: Starting, new_state: Running, cause: AdapterReady, at: "2026-10-22T19:37:36Z" })
fleet: hello-agent [quickstart] state=running
done — the agent ran under governance and stopped cleanly.Two things in that program are worth carrying into your own:
Engine::openowns everything. It creates the state directory if needed, opensstate.db, adopts any agents a previous process left running, and starts the background thread that watches for crashes. All of that stops when you drop the engine.blocking()is the synchronous view. Each method on it runs the matching async method to completion on the engine's own runtime. ThehekmaCLI uses exactly this view. If your program already runs tokio, call the async methods onEnginedirectly instead (see async programs).
Keep the engine open#
The quickstart stops its agent at once. A real host keeps the engine open while agents run, and that is all it takes to get continuous supervision. This program starts an agent that reads one line of input and then reports usage every second, sends it a line, and prints events until a budget breach pauses it:
use std::path::PathBuf;
use std::time::Duration;
use hekma_engine::broadcast::error::RecvError;
use hekma_engine::{AdapterRef, Engine, EngineError, EngineEvent};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let engine = Engine::open(Some(PathBuf::from("/var/lib/my-host")))?;
let host = engine.blocking();
host.register_with_adapter("worker", &AdapterRef::Manifest("/opt/agents/worker".into()))?;
host.set_config("worker", "budget.tokens.cumulative", "3000")?;
let mut events = host.subscribe();
host.start("worker")?;
// The engine holds the agent's stdin for as long as it stays open.
match host.send_input("worker", "hello") {
Ok(()) => println!("sent: hello"),
Err(EngineError::CapabilityUnsupported { .. }) => println!("agent takes no input"),
Err(other) => return Err(other.into()),
}
// Supervision runs on the engine's own threads; this loop only reports.
loop {
match events.recv() {
Ok(EngineEvent::UsageUpdate(u)) => println!(
"usage: +{} in / +{} out (run {})",
u.event.input_tokens, u.event.output_tokens, u.event.run_id
),
Ok(EngineEvent::BudgetBreach(b)) => {
println!("breach: {} of {} tokens, action {:?}", b.observed, b.limit, b.action);
}
Ok(EngineEvent::Transition(t)) => {
println!("state: {} -> {}", t.prior_state, t.new_state);
if t.new_state.as_str() == "paused" {
break;
}
}
Err(RecvError::Lagged(n)) => println!("missed {n} events; resync to catch up"),
Err(RecvError::Closed) => break,
}
}
host.stop("worker", Some(Duration::from_secs(5)))?;
Ok(())
}Built against hekma-engine 0.7.0, with an agent whose manifest declares the interaction capability:
sent: hello
state: registered -> starting
state: starting -> running
usage: +1000 in / +200 out (run run-1792698196454613000-0)
usage: +1000 in / +200 out (run run-1792698196454613000-0)
usage: +1000 in / +200 out (run run-1792698196454613000-0)
breach: 3600 of 3000 tokens, action Pause
state: running -> pausedThe agent's agent.log starts with got: hello, so the input arrived. The budget check ran on the engine's own thread at the third usage event, with nothing in the host doing the arithmetic. Events, resync and diagnostics covers the event stream in full.
The API surface#
Everything a host can call is a method on Engine (async) and on the Blocking view from engine.blocking(), with the same name and arguments. The table groups them by job.
| Group | Methods | What they do |
|---|---|---|
| Open | Engine::open(base), Engine::open_with_diagnostics(base, sink), blocking(), with_diagnostics(sink) | Open an engine on a state directory (None for the default), adopting agents left running. Optionally send the engine's [hekma] diagnostics to your own writer. |
| Register | register(name, kind), register_with_adapter(name, &AdapterRef), remove(name, disposition, force) | Register a built-in kind ("acp", "hermes", "mock") or an AdapterRef::Manifest(path). remove stops a live agent before deleting it, and keeps or deletes the Agent Home per RemoveDisposition. |
| Lifecycle | start, start_detached, stop(name, window), pause, resume | Drive the lifecycle. stop takes an optional graceful window (default 30 seconds). See detached starts for start_detached. |
| Input | send_input(name, text) | Write one line to the agent's stdin, or for an acp agent, start one session/prompt turn and return at once. |
| Config | set_config(name, key, value), effective_config(name, overrides), reveal_secrets(name, overrides) | Write one config key to the instance layer (returns warnings), read the resolved config with each value's source layer, and resolve secret:NAME values to plain text on request. Pass ConfigLayer::empty() for no overrides. |
| Restart policy | set_restart_policy(name, RestartPolicy) | Choose OnFailure (the default) or Never. This is the only way to change the restart policy. |
| Fleet and status | fleet(), fleet_entry(name), list(), instance_status(name), effective_capabilities(name), agent_home(&InstanceName) | Read the rows behind hekma agent list and show: state, usage, budget, metering source and supervision. FleetListing::new(entries) adds the fleet totals. |
| Memory | attach_memory(name, MemoryBackingKind), detach_memory(name), memory_status(name) | Manage the agent's memory backing while it is stopped. attach_memory returns the managed folder's path. |
| Logs | read_agent_log(name), read_agent_log_since(name, cursor), read_raw_output_since(name, cursor) | Read the attributed output.log lines, all or after a byte cursor, or the raw agent.log that a detached agent writes while no engine reads it. Each returns the lines and the next cursor. |
| History | transition_events(name), budget_breach_events(name) | Read the durable transition and breach records from the Agent Home. |
| Events | subscribe(), resync_events(name, ResyncCursor) | Receive events live, and catch up on what you missed. See Events, resync and diagnostics. |
Usage numbers come with the fleet rows: FleetEntry::usage carries cumulative and current-run tokens (and dollars when a rate is set), and FleetEntry::budget carries each limit and what remains. These are the values the CLI prints, with the field names of the list JSON document. The crate also exports the domain types those methods return, such as LifecycleState, TransitionCause, Supervision and Micros, plus render_dollars for formatting money the way the CLI does.
Async programs#
Engine is Send + Sync, so you can keep it in an Arc and call its async methods from your own tokio tasks. The work runs on the engine's runtime; your runtime only awaits it. Two rules apply:
- Do not call
blocking()methods from inside an async task. They run the engine's own runtime on the calling thread, and tokio panics with "Cannot start a runtime from within a runtime". Use the async methods there, or move the call intotokio::task::spawn_blocking. - Drop the engine outside async code. The engine owns a tokio runtime, and tokio panics if a runtime is dropped on an async worker thread. Drop it on a plain thread or with
tokio::task::spawn_blocking(move || drop(engine)).
Several engines#
The engine keeps no global state, so one process can open several engines, each on its own state directory.
Give your host a state directory of its own. Any other engine opened on the same directory, including a single hekma agent list, adopts the agents your engine holds and kills the attached ones when it closes. One engine per state directory explains why and what to do instead.
Detached starts: the host's duty#
start_detached launches an agent that survives the engine. When the engine is dropped, the agent keeps running, and the next engine opened on the same state directory adopts it by process id and start time (see Architecture).
The cost is the gap between engines. While no engine is open there is no crash detection, no budget enforcement and no event delivery, and usage the agent reports in that time is never counted. If you offer detached starts to your own users, say so wherever you offer them, the way hekma agent start --detach does. Supervision and detached runs lists what happens in the gap.
Three more rules carry over from the CLI:
start_detachedfails withEngineError::DetachRefusedfor an engine-observed agent, before anything changes, because its metering proxy lives in the engine and would die with it.- A detached agent's stdin is closed, so
send_inputcannot reach it and an ACP agent cannot run detached. Start those withstartand keep the engine open. - When the engine is dropped, an agent you paused yourself is resumed first, so it is never left frozen with nobody watching. An agent paused by a budget breach stays paused.
A host that keeps one engine open for its whole life rarely needs start_detached. Its main use is surviving a restart of the host itself.
What the engine promises a host#
The engine is built to live inside someone else's process without surprises:
- It never reads your stdin, never prints a prompt, and never checks whether it is attached to a terminal.
- It never writes to your stdout. Its only output is a small set of
[hekma]diagnostic lines on stderr, which you can redirect with a diagnostic sink. - It never changes your process's environment variables, and installs no signal, panic or console handlers.
- It keeps no global state, so several engines can share a process.
- The CLI uses the same public API you do. Every async method has a blocking twin, and the CLI calls only the blocking ones.
Hekma's test suite checks each of these on every change, including a test that runs two engines side by side in one process and checks that neither sees the other's agents or events.
Errors#
Every method returns a typed Result. Nothing panics on a bad name, a wrong state or a failed launch. Three error enums cover most calls:
| Enum | Returned by | Examples |
|---|---|---|
RegistryError | open, register, remove, fleet reads, memory | DuplicateName, NotFound, UnknownAdapterKind, ManifestInvalid, ContractIncompatible, RunningRequiresForce |
EngineError | lifecycle, input, status, events, logs, restart policy | NotFound, InvalidTransition, CapabilityUnsupported, LaunchFailed, DetachRefused, NotRunning, InteractionUnavailable, StopUnconfirmed |
ConfigError | set_config, effective_config, reveal_secrets | UnknownKey (with a suggestion), InvalidValue, InvalidName |
Each variant's Display text is the message the CLI prints, naming the problem and the fix. The CLI maps them to exit codes; for example InvalidTransition is exit code 4 and DetachRefused is 5.
The enums are exhaustive (none is marked #[non_exhaustive]), so a new variant is a breaking change and arrives only in a new minor version of the crate. If you match every variant, a new one is a compile error you fix once; add a _ case if you would rather not. hekma-engine 0.7.0 added these variants since 0.4.0, the previous version on crates.io:
| Variant | When it happens |
|---|---|
EngineError::AcpTurnInFlight | send_input to an acp agent while its previous prompt turn is still running. The first turn is unaffected. |
EngineError::MemoryBackingChangedUnderStart | A memory attach or detach in another process raced this start. Nothing launched; retry the start. |
RegistryError::MemoryBackingRowCommittedDirFailed | A memory attach was recorded but the folder could not be created. |
TransitionCause::DetachedResumed | Not an error, but a new variant of an exhaustive enum: the cause recorded when an engine resumes a detached agent you paused, as it closes. |
StoreError: MemoryBackingKindConflict, MemoryBackingHotSwap, MemoryBackingChangedUnderStart, StateConflict, Busy | State-store errors, seen inside EngineError::Store and RegistryError::Store. Busy means another process held the database for more than 5 seconds. |
Upgrade from 0.4#
hekma-engine went from 0.4.0 straight to 0.7.0 on crates.io; 0.5 and 0.6 were never published. Change the requirement to "0.7" and work through this table:
| What changed | What to do |
|---|---|
New error variants and the DetachedResumed cause (see errors) | Add the new cases to your match, or a _ case. |
New fields on FleetEntry (acp_context_usage, usage_gap, supervision), UsageView and UsageTotals (cached tokens), UsageEvent::cached_tokens, FleetTotals::total_cached_tokens and Rate::cached_micros_per_1m | Only code that builds these structs by hand needs changes. Reading them is unaffected. |
SpawnRecord gained acp_session_id and run_id; SpawnSpec gained pipe_stdout | Only if you implement your own process backend or store. |
ProcessBackend gained take_stdin and take_stdout; StateStore gained set_spawn_run_id (with defaults) and six memory and spawn-record methods (without) | Only if you implement these traits yourself. |
New method read_raw_output_since | Optional: use it to read what a detached agent wrote while no engine was open. |
The acp built-in kind | Register ACP agents with register(name, "acp") and set acp.command. |
| The state database moves to schema 9 on first open | Automatic. An older Hekma refuses to open the upgraded database, so upgrade every program that shares a state directory together. |
Read next#
- Events, resync and diagnostics: the event stream, catching up after a crash, and routing diagnostics.
- Architecture: what the engine does underneath each call.
- Run an ACP agent: the agents that need an embedding host in 0.11.0.
- JSON output: the fields of every event payload.