Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

Ijima is the centralized memory service for the Anima ecosystem — the single source of truth for agentic memory across every harness (pi, Tsume, Sakamoto, Wallace, Dominic, opencode, …). It replaces fragmented, per-harness memory stores with one service every agent reads from and writes to, with per-principal isolation, quantitative access control, and full provenance on every stored fact.

What it does

Ijima serves three related capabilities:

  1. A memory palace — curated memories organized as project → topic rooms, with importance-weighted recall, semantic search, and a diary per agent.
  2. A knowledge graph — subject/predicate/object triples with entity resolution, timelines, and cross-project tunnels.
  3. A session repository + mining pipeline — raw agent session turns flow in, an extraction engine (rules tier, optional LLM tier) proposes candidate memories with confidence scores, and a human/agent review queue promotes the good ones into the palace with provenance intact.

Why it exists

Before Ijima, each harness carried its own memory: pi-mempalace SQLite databases per workstation, ZeroClaw’s Discord brain, ad-hoc files. Memory was fragmented (the same insight saved five times, never reconciled), unauditable (no notion of where a fact came from or how much to trust it), and unsecurable (one all-or-nothing bearer token). Ijima makes memory a service: one instance, many principals, capability-scoped access, and a trust tier on every entry.

Ecosystem position

Ijima is the memory plane under Dominic’s orchestration plane:

  • Schubert provides the capability algebra (Grassmannian Gr(4,8)) that Ijima uses for authorization, proof-carrying GrantTokens, and rate limiting. Ijima is Schubert’s first and most complete consumer.
  • Dominic (meta-orchestrator) dispatches work and federates through Ijima’s control plane; Ijima enforces trust boundaries locally even when Dominic is unreachable.
  • pi connects as a thin client (IJIMA_URL / IJIMA_TOKEN), replacing the pi-mempalace extension.
  • Proserpina supplies the LLM agent surface for the mining tier (proserpina-agent), cross-repo contract-tested.

The workspace

CrateRole
ijima-coreDomain types, Store/KnowledgeGraph traits, capability vocabulary
ijima-serverHTTP daemon (axum), SurrealDB backend, auth, CLI
ijima-clientTyped async HTTP client for harnesses
ijima-minerExtraction engine (rules + LLM tiers)
ijima-pi / integrations/pipi extension (WASM/npm)

Ijima is v0.x: interfaces evolve, breaking changes are announced in the CHANGELOG, and the 0.2.0 “Central Brain” release targets a real single-instance deployment with all workstations as thin clients.

Getting Started

Prerequisites

  • A recent Rust toolchain (see rust-toolchain.toml).
  • (Optional) a model endpoint for the mining LLM tier — DeepSeek by default.
  • (Optional) a Tailscale network for remote thin clients.

Install

Ijima is published on crates.io:

cargo install ijima-server --features "cli,backend-sqlite,embeddings-candle"

The default features give you the daemon, HTTP, Schubert auth, and the SurrealDB backend. Add backend-sqlite if you will import from a legacy pi-mempalace or ZeroClaw database, and embeddings-candle for semantic search (downloads all-MiniLM-L6-v2 on first use).

Start a daemon

ijima serve

The daemon listens on 127.0.0.1:7373, creates a data directory at ~/.ijima (an embedded SurrealDB store + issuer key), and is ready. IJIMA_DIR relocates the data directory; see Installing and Configuring.

Mint a grant

Every request needs a Schubert GrantToken. Mint one for yourself:

ijima token issue --principal elliott \
    --capabilities memory:read,memory:write,knowledge:read,knowledge:write

The token is a base64 GrantToken blob — proof-carrying, signed by the issuer key in the data directory. Store it somewhere safe (a password manager); it is a credential.

First requests

TOKEN="..."  # from the step above

# store a memory
curl -s -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" \
     -d '{"id":"mem_first","content":"Ijima is running","project":"ijima",
          "topic":"notes","source":"Explicit","harness":"Pi",
          "importance":0.5,"created_at":"0"}' \
     localhost:7373/memories

# recall it
curl -s -H "Authorization: Bearer $TOKEN" \
     localhost:7373/memories/mem_first

The response provenance block reports what the daemon saw: source tier, harness, origin instance, authority scope.

From a harness

Rust harnesses use ijima-client:

#![allow(unused)]
fn main() {
let client = ijima_client::Client::new(
    ijima_client::ClientConfig::new("http://127.0.0.1:7373", Harness::Pi)
        .with_token(token),
);
let id = client.store_memory(memory).await?;
}

pi users: the pi integration is a single env pair (IJIMA_URL, IJIMA_TOKEN).

Next steps

The Two-Store Model

Ijima serves two related but distinct stores, unified by a miner that turns one into the other.

1. The Memory Palace

Long-term, curated, semantic memory. Verbatim storage + candle embeddings + cosine search + a temporal knowledge graph (entities and triples with valid_from/valid_to). This is the pi-mempalace model, production-proven, reimplemented in Rust and import-compatible with its schema.

Palace entries always carry provenance: source tier, harness, session, origin instance, and authority scope — so any entry traces back to the conversation that produced it.

2. The Session Context Repository

Raw session transcripts from every harness: every conversation, every agent run, every gateway exchange. Append-only, high-fidelity, uncensored at write time. Not curated, not summarized — the raw ore.

The Miner: ore → metal

The novel capability. Ijima mines raw sessions to extract curated palace entries:

  • Decisions — “we decided to use DeepSeek.”
  • Facts — “Ijima depends on candle.”
  • References — URLs, citations, file paths.
  • Patterns — recurring topics, open threads.

A rules tier runs unconditionally (no model); an optional LLM tier (Proserpina) adds Fact + Pattern roles. Low-confidence extractions stage in a per-namespace review queue rather than auto-archiving. See The Mining Pipeline.

   raw sessions ──▶ [ rules + llm ] ──▶ Auto (palace) / PendingReview (queue)

The raw session always remains in the repository for full-fidelity recall, even after mining.

Namespaces & Multi-Tenancy

Every request to Ijima is scoped to a namespace — the isolation unit. No request spans namespaces; there is no implicit cross-namespace read.

Namespace classes

ClassShapeWho sees it
Personalns_<principal>_privateThe principal only
Sharedns_<org>_shared (e.g. ns_ia_shared)Members (membership-gated)
Importns_import_<source>Imported corpus staging, per source
Global commonsglobalLegacy migration baseline; readable

Routing

  • Omit the namespace → your personal namespace.
  • Pass ?namespace=<ns> → that namespace, subject to checks: another principal’s *_private namespace is always forbidden.
  • Writes require memory:write (or knowledge:write) and namespace eligibility; reads are personal-by-default and explicit otherwise.

Shared-namespace membership (org walls like ns_ia_shared, ns_kellas_shared) is runtime-managed data, not static policy — see the design decision log.

Isolation mechanics

Isolation is enforced at two layers:

  1. API layerresolve_ns rejects foreign private namespaces before the store is touched.
  2. Store layer — SurrealDB record keys are namespaced composites (<namespace>:<memory-id>), so the same logical id can exist in two namespaces without collision. This matters for imports: the same pi-mempalace row imported from two workstations coexists per-source.

Import namespaces

ijima import defaults each source to ns_import_<sanitized-source> — never the global commons. A source name like Laptop 01 becomes ns_import_laptop_01. Import namespaces are staging areas: content lands at the AutoCapture trust tier and is promoted into personal/shared namespaces only after review (see Provenance).

Namespace-aware surfaces

The palace features mirror the memory store’s scoping: rooms and taxonomy are computed per namespace; the knowledge graph is namespaced per triple/entity; sessions and diaries carry their namespace. The /palace/graph and /tunnel traversals operate within one namespace’s view by design.

Provenance & Trust Tiers

Every memory in Ijima carries a provenance block. Provenance is not metadata garnish — it is the basis for trust decisions, promotion, and (eventually) federation conflict resolution.

The provenance fields

FieldMeaning
sourceTrust tier: Explicit, AutoCapture, Mined, or Doctrine
harnessWhich harness wrote it (Pi, Dominic, Wallace, …)
originThe instance that authored the entry
authoritySource-of-truth scope for the entry’s domain
session_idOriginating session, when known

Trust tiers

  • Explicit — an operator or harness deliberately saved it. Highest routine trust.
  • AutoCapture — an automatic hook wrote it. Unverified.
  • Mined — extracted from a session transcript by the miner, carrying a confidence score until reviewed.
  • Doctrine — curated, Git-versioned, PR-reviewed memory mirrored from the repository seed pack. Never written directly by agents.

Trust transitions are themselves capabilities: trust:promote raises an entry’s tier, and cross-tier endorsement/override are progressively more expensive in the capability algebra (see Capabilities). Raising trust costs more than writing at a tier — by construction.

Imports land unverified

ijima import stamps every imported memory origin = <source> and drops the tier to AutoCapture regardless of its original classification — a manual-save row from a workstation’s pi-mempalace arrives as AutoCapture. Imported content is unverified until promoted through the review path. This is deliberate: an import is a claim, not a credential.

Why authority matters

authority records whose fact this is — the local instance, or a remote instance’s scope. In the single-instance present it is uniformly local; when federation lands, per-domain authority scopes drive cross-instance conflict resolution (the instance whose authority scope matches a domain wins that domain’s writes).

Capabilities & GrantTokens

Ijima’s access model is Schubert capability algebra on the Grassmannian Gr(4,8). Capabilities are not role strings — they are Schubert conditions (subspaces), and access decisions are geometry: a grant authorizes an action when the grant’s subspace intersects the capability’s.

The vocabulary

Eleven capabilities, each a partition on Gr(4,8):

CapabilityCodimensionGoverns
memory:read1Read/recall/search memories
knowledge:read1Query the knowledge graph
mining:review2Read the review queue
memory:write2Store/delete memories, diary
knowledge:write2Write triples
session:ingest3Ingest session turns
mining:trigger3+1Trigger mining runs
trust:promote3+1Raise a memory’s trust tier
trust:endorse4+1Cross-tier endorsement
trust:override4+2Authority override (rare)
admin4,4,4,4 (point class)Token admin, status, repos

Write implies read in the geometry: memory:write’s partition contains memory:read’s, so a write grant can also read. A read grant can never write.

GrantTokens

A GrantToken is a compact, proof-carrying, ed25519-signed token that bundles one or more capabilities for one principal:

ijima token issue --principal sara \
    --capabilities memory:read,memory:write,knowledge:read
  • Multi-capability — one grant covers a whole job description; no more per-capability token bundles.
  • Verified geometrically — the daemon decodes the grant, verifies the signature, and checks each request’s capability against the grant’s partition set.
  • Partition-signed — the capability list is inside the signed blob; clients cannot edit grants.
  • Revocable — see Token Management.

Why geometry

The codimension of a capability is a quantitative authorization weight — and it doubles as the rate-limit capacity (intersection-number-scaled token buckets). Low-stakes capabilities (memory:read, codim 1) are cheap; trust-flow capabilities (trust:override) are expensive; admin is the point class. The geometry of access is also the geometry of throughput, and trust-flow costs more than access — deliberately.

The vocabulary is declarative (policy.toml in the daemon crate); adding a capability is a policy edit plus a partition assignment, not a code change.

Sessions & the Miner

The session repository is Ijima’s raw-material intake; the mining pipeline turns it into curated memory. This is Ijima’s most distinctive loop: raw conversation in one end, reviewed, provenance-carrying memory out the other.

The flow

harness ──POST /sessions/:id/turns──▶ session repository (verbatim)
                                          │
                            mining:trigger│
                                          ▼
                                    extraction engine
                              (rules tier → LLM tier)
                                          │
                       Auto-confidence ───┼─── PendingReview
                       (auto-filed)            │
                                               ▼
                                        review queue
                                               │ mining:review
                                               ▼
                                     accept → palace (Mined tier)
                                     reject → archived
  1. Ingest (session:ingest) — harnesses stream turns verbatim. No filtering, no interpretation: the repository is the audit record.
  2. Extract (mining:trigger) — the extraction engine runs over a session. The rules tier (deterministic, cheap) catches decisions, TODOs, and entity mentions; the LLM tier (optional, via a Proserpina HTTP agent) proposes richer candidates with confidence scores.
  3. Route by confidence — high-confidence extractions file automatically as Mined memories; the rest wait in the review queue.
  4. Review (mining:review) — a human or agent reviews the queue, accepting (→ palace, provenance Mined + source session) or rejecting (→ archived, kept for the record).

Why sessions stay verbatim

The repository is deliberately unprocessed: mining proposals can be rejected, improved, and re-run, but the source transcript is immutable evidence. Provenance on every mined memory points back at the session and turn range it came from.

Trust flow

Mined memories enter at the Mined tier — above nothing, below Explicit. Promoting them (or any memory) to higher trust is a separate, deliberate act gated by trust:promote. The pipeline never silently raises trust.

See The Mining Pipeline for operation, and the miner architecture ADR for the tier design.

Installing and Configuring

Configuration precedence

Ijima resolves settings in a strict layer order — later wins:

defaults  <  ijima.toml file  <  environment variables  <  CLI flags

Config file discovery

The daemon looks for ijima.toml in order:

  1. $IJIMA_CONFIG — explicit pointer (a missing or malformed file at an explicit pointer is a hard error, not a silent fallback)
  2. $IJIMA_DIR/ijima.toml
  3. /etc/ijima/ijima.toml

Config keys

# /etc/ijima/ijima.toml
host = "127.0.0.1"
port = 7373
data_dir = "/var/lib/ijima"
issuer_key = "/var/lib/ijima/issuer.key"
rate_base = 10
rate_multiplier = 1.0
embedding_model = "sentence-transformers/all-MiniLM-L6-v2"

Unknown keys are ignored (forward compatibility).

Environment variables

VariablePurpose
IJIMA_DIRData directory (default ~/.ijima)
IJIMA_CONFIGExplicit config file path
IJIMA_HOST / IJIMA_PORTBind address overrides
IJIMA_KEYIssuer key path override
IJIMA_RATE_BASE / IJIMA_RATE_MULTIPLIER / IJIMA_RATE_DISABLERate-limit tuning
IJIMA_TLS_CERT / IJIMA_TLS_KEYPEM paths for the tls feature
IJIMA_EMBED_MODEL / IJIMA_EMBED_REVISIONEmbedding model pinning
IJIMA_LLM_MODEL / IJIMA_LLM_BASE_URL / IJIMA_LLM_API_KEYMining LLM tier endpoint
IJIMA_INSTANCE_ID / IJIMA_INSTANCE_ROLE / IJIMA_INSTANCE_SCOPESFederation instance identity
IJIMA_LOGLog filter

Client-side (thin clients and the CLI’s remote commands): IJIMA_URL (default http://127.0.0.1:7373) and IJIMA_TOKEN (the bearer grant).

Feature selection at install time

cargo install ijima-server \
    --features "cli,backend-sqlite,embeddings-candle,mining,tls"

See Feature Flags for the full matrix. The default set (std,http,server-auth,backend-surreal) runs a production daemon; add backend-sqlite only for one-time imports (it exists to read legacy databases).

Running the Daemon

Foreground

ijima serve                     # 127.0.0.1:7373, data at ~/.ijima
ijima serve --host 0.0.0.0 --port 7373
IJIMA_DIR=/var/lib/ijima ijima serve

systemd (production)

The repository ships a hardened unit at deploy/ijima.service plus a commented deploy/ijima.toml.example:

sudo install -m644 deploy/ijima.toml.example /etc/ijima/ijima.toml
sudo install -m644 deploy/ijima.service /etc/systemd/system/
sudo systemctl enable --now ijima

The unit runs with a dedicated user, ProtectSystem=strict, ReadWritePaths=/var/lib/ijima, and Restart=on-failure.

Observability

  • GET /health — liveness (no auth).
  • GET /status — memory/namespace/entity/triple counts, version, start time, uptime (admin).
  • Structured logs via tracing (IJIMA_LOG filter; RUST_LOG also respected by convention).

TLS

With the tls feature, set IJIMA_TLS_CERT and IJIMA_TLS_KEY (PEM paths) — the daemon binds HTTPS via axum-server/rustls. On a private Tailscale network, tailscale serve in front of plain HTTP is the documented alternative.

Restart semantics

The SurrealDB (surrealkv) engine holds a directory LOCK while open. Dropping the in-process handle does not release it synchronously — background engine tasks must wind down. This is invisible across process boundaries (the OS releases the lock at exit), which is the normal restart path for the daemon. Only embedders opening/closing the same data directory sequentially inside one process need the spawn-and-yield pattern (documented on SurrealStore::open_persistent).

Upgrades

Stop the daemon, replace the binary, start. The store’s schema is defined idempotently at open (DEFINE TABLE IF NOT EXISTS / DEFINE INDEX IF NOT EXISTS), so minor upgrades with an unchanged on-disk layout need no migration step. Pre-0.2 development databases (pre-namespaced record keys) should be re-imported rather than carried forward — see Importing Legacy Corpora.

Token Management

Grants are credentials. This chapter is the operator’s lifecycle guide.

Issuing

# a full personal grant
ijima token issue --principal elliott \
    --capabilities memory:read,memory:write,knowledge:read,knowledge:write

# a machine feed grant (session ingest + mining only)
ijima token issue --principal minoru \
    --capabilities session:ingest,mining:review --json

# an operator/admin grant (rare; the point class)
ijima token issue --principal ops --capability admin --json

--json emits {token, principal, capabilities, public_key} for scripting. --capabilities takes a CSV (multi-capability GrantToken); --capability takes a single value. The grant is signed by the issuer key in the data directory (IJIMA_KEY / issuer_key config to relocate) — tokens minted against one key do not verify on a daemon with another key.

Issue narrow grants: a dispatcher that only ingests sessions gets session:ingest and nothing else. The geometry enforces what the grant says, not what the principal “should” have.

Deploying to clients

Thin clients need two env vars:

export IJIMA_URL="http://ijima.tailnet:7373"
export IJIMA_TOKEN="<grant blob>"

The pi extension, ijima-client, and the CLI’s remote subcommands all honor them.

Revoking

The kill-switch for leaked or rotated credentials:

ijima token revoke --token "<bearer>" \
    --url http://127.0.0.1:7373 --auth "<admin-bearer>" \
    --reason "leaked in CI log"
  • Revocation is store-backed and survives restarts: the daemon persists a SHA-256 hash of the bearer and hydrates an in-memory set at boot. Raw bearer values never touch the store, logs, or backups.
  • The check composes with signature verification: a revoked bearer is rejected even though its signature is valid.
  • Past revocations: ijima token revocations --auth "<admin-bearer>" (admin), oldest first.

Rotation practice

  • Prefer revoke + fresh issue over long-lived shared grants.
  • Issuer-key rotation (replace the key file, restart) invalidates all grants at once — the emergency lever for key compromise, not routine rotation.
  • GrantToken expiry is upstream-gated on Schubert 0.5 (expires_at + nonce in the signed blob); until then, revocation is the routine deprovisioning path.

Importing Legacy Corpora

ijima import streams an external SQLite corpus into a running daemon over HTTP — the files never leave the workstation, the daemon does the storing. This is the 0.2.0 path for consolidating pi-mempalace (and ZeroClaw) history into the central instance.

Usage

ijima import mempalace --db ~/.pi/agent/mempalace/memories.db \
    --source "elliotthall-laptop"

ijima import zeroclaw --db ~/zeroclaw/brain.db --source "zeroclaw-archive"
FlagMeaning
--db PATHSource SQLite database
--source NAMEProvenance origin stamp + namespace derivation
--namespace NSOverride the target namespace
--url URLDaemon (default $IJIMA_URL or 127.0.0.1:7373)
--token TOKENA memory:write grant (default $IJIMA_TOKEN)

What the importer does

  1. Reads the source rows (memories; pi-mempalace knowledge-graph rows arrive as triples).
  2. Retags provenance: origin = <source>, trust tier dropped to AutoCapture regardless of original classification. Harness provenance is preserved (Pi for mempalace, Other for ZeroClaw).
  3. Routes into ns_import_<sanitized-source> (e.g. Laptop 01ns_import_laptop_01) — never the global commons. --namespace overrides for deliberate shared targets.
  4. Dedups: every memory is pre-checked via POST /memories/check (content-hash) before storing. The same memory saved on two workstations stores once per source namespace; re-running an import is idempotent.
  5. Reports per source:
{ "attempted": 1284, "added": 1190, "deduped": 91, "skipped": 3 }

skipped counts per-row failures — one bad row never aborts the run.

After import

Import namespaces are staging. Review the content and promote what you trust into personal or shared namespaces (trust:promote); the origin stamp (elliotthall-laptop) travels with every promoted memory, so the workstation trail survives.

The older migrate path

ijima migrate --palace <db> is the pre-HTTP one-shot local import (runs against the daemon’s own data directory, no HTTP). Prefer import — it works against remote daemons, dedups per-source, and tags provenance properly.

The Mining Pipeline

Ijima’s novel capability: turning raw session transcripts into curated memory palace entries with full provenance.

Tiers

  • Rules tier (always on, no model): deterministic extraction of Decisions (“we decided to…”) and References (URLs / scheme: links). Fast, free, side-effect-free.
  • LLM tier (optional, mining feature): Proserpina-backed Fact and Pattern roles. Single-shot per role (no panel cross-examination in v0). Emits one JSON object per line: {"content","project","topic","confidence"}.

Configure the LLM tier with IJIMA_LLM_BASE_URL / IJIMA_LLM_MODEL / IJIMA_LLM_API_KEY. When model/key are unset, mining runs rules-only.

Routing

  • Auto extractions archive straight to the palace (content-hash dedup applies).
  • PendingReview extractions stage in a per-namespace review queue. Confidence ≥ 0.85 overrides a role’s default to Auto — a high-confidence fact auto-archives.

Trigger

# Mine a session, then review what landed in the queue.
curl -X POST http://127.0.0.1:7373/sessions/sess_1/mine \
  -H "authorization: Bearer <mining:trigger-token>"

curl http://127.0.0.1:7373/mining/queue \
  -H "authorization: Bearer <mining:review-token>"

curl -X POST http://127.0.0.1:7373/mining/queue/<id>/accept \
  -H "authorization: Bearer <mining:review-token>"

Architecture notes

Extraction is pure and synchronous; a separate async ingest step writes to the store. Because the Proserpina HttpAgent::respond blocks on its own tokio runtime, the daemon runs the sync mine_all pass inside spawn_blocking — owning the concrete (Send) agent and coercing to &mut dyn Agent inside the closure. See docs/adr/miner-architecture.md.

pi Thin-Client Integration

pi (the coding-agent harness) talks to Ijima as a thin client — no local store, no local daemon. The extension is integrations/pi (npm), built from the ijima-pi crate compiled to WASM.

Setup

Two environment variables replace the old four-token pi-mempalace bundle:

export IJIMA_URL="http://ijima.tailnet:7373"
export IJIMA_TOKEN="<grant blob>"

Mint the grant on the daemon:

ijima token issue --principal pi-workstation \
    --capabilities memory:read,memory:write,knowledge:read,knowledge:write

What the extension provides

The pi tool surface maps onto Ijima routes:

pi toolIjima surface
Memory search / save / dedup-check/memories, /memories/search, /memories/check
Knowledge add / query / timeline/kg/*
Rooms, taxonomy, palace graph/rooms, /taxonomy, /palace/graph
Diary write/read/diaries

Requests are built and parsed in WASM (the ijima-pi crate’s request/response types), so the wire contract is compiled once and shared — the pi process itself does no JSON hand-rolling.

Why thin clients

The 0.2.0 topology decision: all workstations are thin clients of one central instance (the “Central Brain” deployment). Local memory state on workstations means fragmentation again — the thing Ijima exists to end. Full local instances with checkpoint sync (satellites) are the 0.3 design, not the 0.2 reality.

Replacing pi-mempalace

If the workstation has an existing pi-mempalace database, import it once:

ijima import mempalace --db <memories.db> --source "$(hostname)"

then point pi at Ijima with the env vars above and retire the local store.

Deploying on a Server

The reference deployment is a single central instance on a trusted server, with every workstation as a thin client. This chapter condenses the full runbook (docs/deploy/laniakea.md in the repository) to the essentials.

Topology

workstations (pi, agents)          laniakea (or any always-on host)
┌────────────────────┐   HTTP    ┌──────────────────────────┐
│ IJIMA_URL=...      │──────────▶│ ijima serve (systemd)    │
│ IJIMA_TOKEN=...    │  tailnet  │ /var/lib/ijima (NVMe)    │
└────────────────────┘           │ zpool/backups (nightly)  │
                                 └──────────────────────────┘
  • Network: Tailscale; the daemon binds the tailnet interface. TLS via tailscale serve (terminates on the tailnet’s certs) or the tls feature directly.
  • Storage: data directory on fast local disk (SurrealDB/surrealkv); nightly snapshots to bulk storage.

Provision checklist

  1. Install the binary (see Getting Started) with cli,backend-sqlite,embeddings-candle,mining,tls as needed.
  2. /etc/ijima/ijima.toml — host/port/data_dir/issuer_key (copy from deploy/ijima.toml.example).
  3. deploy/ijima.service → systemd; systemctl enable --now ijima.
  4. Mint grants per principal (operators, harnesses, machine feeds) — Token Management.
  5. curl .../health liveness; GET /status (admin) for counts.
  6. Import workstation corpora — Importing Legacy Corpora.
  7. Point each workstation’s IJIMA_URL/IJIMA_TOKEN at the instance.

Backup & restore

The store is a directory. The drill:

systemctl stop ijima
zfs snapshot zpool/backups/ijima@$(date +%F)   # or rsync the directory
systemctl start ijima

Restore = stop, replace the directory, start. Run the drill once before trusting it.

Upgrades

systemctl stop ijima && <install new binary> && systemctl start ijima

Schema definitions are idempotent at open. Check the CHANGELOG for on-disk-layout notes before skipping multiple minors.

First-day verification

After provisioning: import one real workstation, run a pi session against the central instance for a day, and confirm /status counts grow. The deployment isn’t real until a harness has lived on it.

Feature Flags

Ijima is additive-feature-gated: compose the build with the surface you need. The daemon crate’s matrix:

FeatureEnables
stdstd prelude integration
httpThe axum HTTP daemon
server-authSchubert auth (GrantTokens, policy, verifier)
backend-surrealSurrealDB store (embedded kv-mem / surrealkv) — the primary backend
backend-sqliteMigration-only SQLite readers for pi-mempalace / ZeroClaw imports
rate-limitSchubert intersection-number token buckets
cliThe ijima binary (clap + reqwest + ijima-client)
embeddings-candleLocal embeddings (all-MiniLM-L6-v2 via candle/HF)
miningExtraction pipeline (ijima-miner + proserpina-agent HTTP LLM tier)
tlsaxum-server/rustls HTTPS (IJIMA_TLS_CERT/IJIMA_TLS_KEY)
federation/federation/* control-API scaffold + instance identity

Defaults: std,http,server-auth,backend-surreal — a production daemon without the CLI. (The published CLI binary ships cli plus the optional surfaces.)

Companion crates gate independently: ijima-client’s remote (reqwest transport) and std; ijima-core’s serde and federation.

Choosing

  • Server: default + cli + embeddings-candle (+ mining, tls, backend-sqlite during migration).
  • Embedded library use (tests, in-process): backend-surreal without http/server-auth — an unauthenticated in-process store.
  • Thin clients: ijima-client alone.

Notes

  • backend-sqlite exists to read legacy databases once; it is not a runtime store backend.
  • server-auth off means unauthenticated single-process mode — for tests and embedded use only, never a networked daemon.
  • The pi extension builds from ijima-pi to WASM — see integrations/pi.

HTTP API Overview

Ijima’s stable contract is a REST/JSON surface; every route is guarded by a Schubert capability check and scoped to a namespace. The typed client is ijima-client. Namespace-sensitive routes accept ?namespace=<ns> (omit for the caller’s personal namespace; foreign *_private namespaces are always rejected).

Route map (selected)

MethodPathCapability
GET/health(none)
GET/statusadmin
POST/memories[?namespace=]memory:write
GET/memories?namespace=&limit=memory:read (browse)
GET/memories/:id?namespace=memory:read
DELETE/memories/:id?namespace=memory:write
POST/memories/check?namespace=memory:read (dedup pre-check)
POST/memories/searchmemory:read
POST/memories/:id/promotetrust:promote
GET/wakeupmemory:read (top-N wake-up context)
GET/rooms, /taxonomy, /palace/graph, /tunnelmemory:read
POST/kg/triplesknowledge:write
GET/kg/entities/:id, /kg/timeline/:entityknowledge:read
POST/sessions, /sessions/:id/turns, /sessions/:id/endsession:ingest
POST/sessions/:id/minemining:trigger
GET/mining/queue, decisions (accept/reject)mining:review
POST/diariesmemory:write
GET/repos/resolve?path=memory:read (RepoDirectory)
POST/tokens/revokeadmin
GET/tokens/revocationsadmin
GET/federation/state, routed-write, conflict-signalfederation feature

The full table (with the store method each route maps to) lives in the daemon crate’s api module doc comment.

Authentication

One header: Authorization: Bearer <GrantToken>. The grant bundles the principal’s capabilities; the daemon verifies the signature, checks revocation, and evaluates each request’s capability geometrically (see Capabilities).

Conventions

  • Errors are JSON with a stable shape; 403 means the grant lacks the capability (or the namespace is off-limits), 401 means the bearer failed verification/revocation, 409 duplicate content on store.
  • Provenance fields are accepted on write and echoed on read — the daemon stamps defaults (created_at, instance identity) where absent.
  • All list endpoints are limit-bounded.

Client

For harnesses, depend on ijima-client (typed async HTTP) rather than hand-rolling requests — see The Client Crate.

The Client Crate

ijima-client is the typed async HTTP client for harnesses and tools. It is transport-thin: it builds requests, attaches the bearer, decodes responses — and nothing else. All policy lives in the daemon.

Setup

[dependencies]
ijima-client = "0.1"
#![allow(unused)]
fn main() {
use ijima_client::{Client, ClientConfig};
use ijima_core::harness::Harness;

let client = Client::new(
    ClientConfig::new("http://ijima.tailnet:7373", Harness::Pi)
        .with_token(token),
);
}

ClientConfig is cloneable; with_token accepts the raw grant (the Bearer prefix is added internally).

Surface (selected)

#![allow(unused)]
fn main() {
// memories
client.store_memory(memory).await?;                  // personal ns
client.store_memory_in("ns_import_laptop", m).await?; // explicit ns
client.recall_memory("mem_x", Some("ns_ia_shared")).await?;
client.delete_memory("mem_x", None).await?;
client.search_memories(&query, Some("ns_ia_shared")).await?;
client.check_duplicate("content", Some(ns)).await?;   // dedup pre-check
client.import_memories(ns, memories).await?;          // dedup-checked bulk

// knowledge graph
client.add_triple(triple).await?;
client.query_entity("amari", Some(ns)).await?;

// sessions & mining
client.ingest_turn(session_id, turn).await?;
client.trigger_mining(session_id).await?;

// palace surfaces
client.list_rooms(Some(ns)).await?;
client.taxonomy(Some(ns)).await?;
client.palace_graph(Some(ns)).await?;
}

import_memories returns ImportCounts { attempted, added, deduped, skipped } — the WS2 import loop in a single call.

Errors

Everything surfaces as IjimaError: Transport (HTTP failure, carrying status + body detail), Store, or domain errors. A 404 from recall_memory maps to Ok(None) — absence is not an error.

Feature notes

  • remote (default) brings reqwest; disable for an in-process stub in tests.
  • The client identifies its harness in provenance fields — set the real one; the daemon records it on every write.
  • For pi specifically, the compiled extension (integrations/pi) wraps this surface for WASM — pi processes never hand-roll JSON.

CLI Reference

The ijima binary (build with the cli feature) is the operator’s surface: daemon control, token lifecycle, imports, migration.

ijima serve

Runs the HTTP daemon. Flags override env/config: --host, --port, --data-dir, --issuer-key. See Running the Daemon.

ijima token

ijima token issue --principal NAME (--capability CAP | --capabilities A,B,C) [--json]
ijima token revoke --token "<bearer>" --url URL --auth "<admin>" [--reason "why"]
ijima token revocations --auth "<admin>" [--url URL]
  • issue runs offline — it signs with the issuer key in the local data directory, so run it where the daemon’s key lives (or point IJIMA_DIR/IJIMA_KEY at it).
  • revoke/revocations are remote calls to a running daemon’s admin routes (--url defaults to $IJIMA_URL).

See Token Management.

ijima import

ijima import (mempalace|zeroclaw) --db PATH --source NAME
             [--namespace NS] [--url URL] [--token TOKEN]

Streams a legacy SQLite corpus into a running daemon with provenance retagging, per-source namespaces, and dedup pre-checks. Defaults from $IJIMA_URL / $IJIMA_TOKEN. See Importing Legacy Corpora.

ijima migrate

ijima migrate [--palace PATH] [--brain PATH] [--embed] [--namespace NS]

The older one-shot local import (writes into the daemon’s own data directory, no HTTP). Prefer import.

ijima export

ijima export --out dump.sql

Dumps the store as SurrealDB SQL (backup/migration aid).

Exit codes and errors

Errors print ijima: <message> on stderr and exit non-zero. Remote commands surface HTTP detail verbatim — a 403 from the daemon names the missing capability.

Security Considerations

Ijima’s access model is Schubert capability algebra on the Grassmannian Gr(4,8) — quantitative, geometry-based authorization, not ad-hoc role checks.

Identity & authentication

  • Principals are operators or harnesses; a request is always (principal, harness, action).
  • GrantTokens are proof-carrying, ed25519-signed bearer credentials bundling one or more capabilities; the capability list is inside the signed blob. Verified by the daemon’s GrantVerifier.
  • Write implies read in the geometry (memory:writememory:read); the converse never holds.
  • Revocation: store-backed SHA-256 hash list checked after signature verification; survives daemon restarts; raw bearer values never touch the store, logs, or backups. Issuer-key rotation remains the emergency lever (invalidates every grant at once).

Authorization

Eleven capabilities as Schubert partitions; the intersection number (codimension) of a capability is both its authorization weight and its rate-limit capacity. memory:read (codim 1) → 1× throughput; memory:write (2) → 2×; admin (the point class σ₄₄₄₄, codim 16) → 16×. The geometry of access maps to the geometry of throughput.

Trust tiers & trust-flow

Trust-tier transitions are themselves capabilities (the provenance-tier model): trust:promote gates tier promotion — raising trust is costlier than writing at a tier. trust:endorse and trust:override (default-deny) cover cross-tier endorsement and authority override. The same geometric policy therefore governs who may access and how trust may flow. Imported content lands AutoCapture regardless of origin claims — an import is a claim, not a credential.

Boundaries

  • Namespace isolation is enforced at two layers: resolve_ns at the API (foreign *_private is always forbidden) and namespaced record keys at the store (same logical id in two namespaces cannot collide or cross-read).
  • Promotion is the single redaction boundary — the one place content filtering (secret stripping) happens. Personal storage is always verbatim.
  • Provenance (origin instance + authority scope + source tier) on every memory is the foundation for federation cross-talk policies and context-poisoning protection.

Known limitations (honest)

  • v0.2.0 is single-instance: federation routes exist as a scaffold; cross-instance enforcement is 0.3+.
  • Rate limiting is per-principal token-bucket; it is not a DoS shield — deploy behind a real proxy on untrusted networks.
  • TLS is opt-in (tls feature); on a private Tailscale network, plain HTTP behind tailscale serve is the documented default.
  • GrantToken expiry is upstream-gated on Schubert 0.5; until adopted, revocation is the routine deprovisioning path.
  • Context-poisoning protection is designed but not yet implemented.

See docs/DESIGN.md and docs/adr/ (grant-token-migration, token-revocation, provenance-tier-model).

Architecture

The workspace

ijima-core      domain types + Store/KnowledgeGraph traits + capability vocab
ijima-server    axum daemon, SurrealDB backend, Schubert auth, CLI
ijima-client    typed async HTTP client (thin clients)
ijima-miner     extraction engine (rules + LLM tiers)
ijima-pi        pi extension types (compiled to WASM for integrations/pi)

Dependencies flow one direction: server → core/miner/client; miner → core (+ proserpina-agent for the LLM tier); client → core. ijima-core has no HTTP, no database, no async runtime beyond trait definitions — the domain is pure.

Store backends

  • SurrealDB (backend-surreal) — the primary backend. Embedded engines: kv-mem (tests, ephemeral) and surrealkv (persistent single-file). Record keys are namespace-composite (<ns>:<id>); tables are defined idempotently at open; every filtered column has a DEFINE INDEX (SurrealDB does not auto-index).
  • SQLite (backend-sqlite) — migration-only readers for the legacy pi-mempalace / ZeroClaw corpora. Never a runtime backend.

The daemon

serve builds the router once at boot: middleware order is authentication (bearer → GrantToken verify → revocation check) → rate limiting (intersection-number token buckets) → capability check per-route → resolve_ns → handler. All handlers are thin: they map HTTP onto Store/KnowledgeGraph trait calls. The mining feature adds the extraction pipeline as an in-process stage over the session repository.

Deployment modes

  • Daemon + thin clients (the 0.2.0 “Central Brain” topology): one instance on an always-on host; workstations point IJIMA_URL at it.
  • Embedded in-process: build backend-surreal without http/server-auth for tests and embedding — an unauthenticated direct store.
  • Satellites (0.3 design): full local instances with checkpoint sync to the center; the federation control API scaffold is the seed.

Provenance by construction

Provenance is not a bolt-on: Memory is content + provenance in the domain type. Every write path (HTTP, import, mining, doctrine ingest) must produce a full provenance block; every read path carries it back. This is what makes trust tiers and (future) federation conflict resolution enforceable at the type level.

Decision Log (ADRs)

Architecture decisions live as ADRs in docs/adr/ in the repository. Index with one-line outcomes:

ADRDecision
grant-token-migration.mdAdopt Schubert 0.4 GrantTokens (multi-capability, partition-signed) as the sole bearer format; delete the duplicated wire codec; admin via geometry (point class), not string equality
token-revocation.mdStore-backed SHA-256 bearer-hash revocation list (no raw bearers at rest); checked after signature verification; expires are upstream (Schubert 0.5) — revocation and expiry are complementary
provenance-tier-model.mdMemorySource trust grades map to Schubert codimensions; trust transitions are capabilities; imports land AutoCapture
miner-architecture.mdTwo-tier extraction (deterministic rules + optional LLM via proserpina-agent); confidence-routed to auto-file or review queue
compaction-recovery.mdSession compaction keeps recoverable turn history for re-mining
federation-control-api.md/federation/* control scaffold with instance identity (IJIMA_INSTANCE_*); boundary enforcement staged for 0.3

Standing decisions recorded elsewhere

  • Thin clients in 0.2.0, satellites in 0.3 — all workstations point at the central instance; local instances with checkpoint sync are the next design (docs/plans/).
  • Membership-in-store over policy-TOML grants for shared namespaces — mutable at runtime without redeploying.
  • Config precedence defaults < file < env < CLI, and an explicit $IJIMA_CONFIG pointing at a missing/malformed file is a hard error.
  • Announcements stay manual — the reusable auto-announce workflow will be proven on another project first.

Roadmap

Shipped:

  • v0.1.0 (2026-08-10) — the library: two-store model, Schubert capability auth, SurrealDB backend, mining pipeline, pi extension, crates.io publication of core/server/miner/client.
  • v0.2.0 “Central Brain” (in progress) — the deployment release: config-file layer + deploy kit (WS1), GrantToken migration, token revocation, multi-source import over HTTP (WS2), namespace membership (WS3), Proserpina agent surface (WS0), dependency sweep incl. surrealdb 3.

Next (0.3 horizon):

  • Satellite sync — full local instances with checkpoint export/push to the center (the WS6 design seed); the federation control API grows into enforcement.
  • Batch ingest (turns:batch) for machine feeds (Minoru mining, Quantizon experiments) and scheduled mining (CLI + systemd timer) ride the 0.2.x line.
  • Schubert 0.5 adoption — GrantToken expiry + nonce; reconciliation with instance-side revocation as defense in depth.
  • Block↔memory promotion boundary — the doctrine note (Lonis Block kinds × promotability × trust tiers) that Wallace and Ijima will implement against.

Long game:

  • Context-poisoning protection (defending trusted-tier doctrine going pathological) — designed, gated on a real incident report.
  • Networked instances / federation cross-talk policies.
  • The doctrine-authority question: is Ijima the authoritative doctrine store, or an opt-in doctrine-health contract?

The authoritative, continuously-updated version lives in the repository: docs/ROADMAP.md.

Basic Usage

A round-trip: store, search, mine, promote. Assumes a running daemon (ijima serve) and a grant issued via ijima token issue.

Store and recall

TOKEN="..."

curl -s -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" \
     -d '{"id":"mem_1","content":"Amari is the flagship math library",
          "project":"amari","topic":"project-context",
          "source":"Explicit","harness":"Pi",
          "importance":0.7,"created_at":"0"}' \
     localhost:7373/memories

curl -s -H "Authorization: Bearer $TOKEN" localhost:7373/memories/mem_1

Dedup pre-check

curl -s -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" \
     -d '{"content":"Amari is the flagship math library"}' \
     localhost:7373/memories/check
# {"duplicate":"mem_1"}

From Rust

#![allow(unused)]
fn main() {
let client = Client::new(
    ClientConfig::new("http://127.0.0.1:7373", Harness::Pi).with_token(token),
);

client.store_memory(memory).await?;
let hits = client
    .search_memories(&SearchQuery { text: "flagship".into(), ..Default::default() }, None)
    .await?;
}

Mine a session

# ingest a turn
curl -s -H "Authorization: Bearer $INGEST_TOKEN" -H "content-type: application/json" \
     -d '{"turn_index":0,"role":"User","content":"we decided to use surrealdb","timestamp":"..."}' \
     localhost:7373/sessions/sess_1/turns

# trigger mining (mining:trigger grant)
curl -s -X POST -H "Authorization: Bearer $MINER_TOKEN" \
     localhost:7373/sessions/sess_1/mine

# review the queue (mining:review grant)
curl -s -H "Authorization: Bearer $REVIEW_TOKEN" localhost:7373/mining/queue

Accepting a queued extraction files it as a Mined memory with the source session in its provenance; rejecting archives it.

Promote

An imported or mined memory earns trust explicitly:

curl -s -X POST -H "Authorization: Bearer $TRUST_TOKEN" \
     localhost:7373/memories/mem_imported_1/promote

Full walkthroughs

Multi-Source Import

Consolidating two workstations’ pi-mempalace corpora into one central daemon — the WS2 workflow end to end.

Scenario

  • elliotthall-laptop and kaiizen each have a pi-mempalace memories.db; the same insight was sometimes saved on both.
  • The central daemon runs at ijima.tailnet:7373.

1. Mint an import grant (once)

ijima token issue --principal importer \
    --capabilities memory:read,memory:write --json
export IJIMA_URL="http://ijima.tailnet:7373"
export IJIMA_TOKEN="<grant>"

2. Import each source

On each workstation (scp the db to the server, or run locally with IJIMA_URL pointed over the tailnet):

ijima import mempalace --db ~/.pi/agent/mempalace/memories.db \
    --source "elliotthall-laptop"
# ijima: import `elliotthall-laptop` complete — 1190 added, 91 deduped, 3 skipped

ijima import mempalace --db /srv/kaiizen/memories.db --source "kaiizen"
# ijima: import `kaiizen` complete — 940 added, 150 deduped, 0 skipped

Each lands in its own staging namespace — ns_import_elliotthall_laptop and ns_import_kaiizen — with origin stamped per source and every memory at the AutoCapture tier, even rows the source called manual-save.

The deduped counts are per-source content-hash collisions (the same memory saved twice on one machine). Cross-source overlap is preserved deliberately: both namespaces keep their copy, each tagged with its origin, until you review.

3. Inspect per-source overlap

curl -s -H "Authorization: Bearer $TOKEN" \
     "localhost:7373/memories?namespace=ns_import_kaiizen&limit=50" \
  | jq '.[] | select(.origin=="kaiizen") | .content'

4. Promote what you trust

Review in the staging namespaces, then promote winners into personal or shared namespaces — trust:promote grant required:

curl -s -X POST -H "Authorization: Bearer $TRUST_TOKEN" \
     "localhost:7373/memories/mem_9812/promote?namespace=ns_import_kaiizen"

The origin stamp travels with the promoted memory — the workstation trail survives promotion.

5. Re-run safety

Imports are idempotent: re-running the same command reports the previous added count as deduped and adds nothing. Schedule freely.