DocsBuildAPI ReferenceSDK Reference

OpsGraph Deep Dive — Profiles, Confirmation, Analytics, Monitoring, and Access Broker

How HUMΛN turns inventory into confirmed operational knowledge — detects latent failure without paging on green servers — and remediates only under explicit Delegation.

Mechanism walkthrough

  1. Connect — org-resolved credentials / collectors (host agents, cloud APIs, AI usage APIs).
  2. Discover — inventory + fingerprint match against profile packs.
  3. Propose — proposeSuccessProfile always emits knowledge_class: proposed (never canonical).
  4. Human confirm — confirmSuccessProfile promotes to canonical; unconfirmed stays proposed.
  5. Instrument — collectors emit observations; analytics modules evaluate platform telemetry (kb/167), not an OpsGraph-private silo.
  6. Ask — Companion, MCP, and CLI share analytics contracts (bundle-owned HTTP) and Kind A monitoring routes.
  7. Detect — threshold + baseline probes with streak hysteresis → latent-failure findings.
  8. Correlate — one cp_incidents row per correlation key (cluster blast-radius preference).
  9. Triage / alert — HUMΛN-native read-only triage + environment-aware routing (monitoring alone still does not mutate infrastructure).
  10. Remediate (Phase 3) — typed action propose → HITL approve (mints scoped Delegation) → OpsNode constrained dispatch → product-outcome recovery verification; optional short-lived SSH cert via HUMΛN CA.

Profile packs are contributed via generic opsgraph_profile_packs (module + export_name). Zero literal pack/bundle-id branches in API services/routes.

Pack catalog (Phase 1.5)

First-party profile ids: linux_host; nginx; httpd; pound; varnish; haproxy; redis; memcached; mysql; postgres; cratedb; rabbitmq; kafka; aws; digitalocean; azure; gcp; openai; anthropic; google_gemini.

See docs/guides/opsgraph/pack-catalog.md (links cloud/AI setup guides).

Phase 3 — Access Broker & delegated remediation

OpsGraph promises evidence of product health while preserving operator control: discovery never silently grants remediation authority. Phase 3 makes that boundary operational for VM fleets via OpsNode (AgentPassport), typed capability_nodes, Delegation-gated dispatch, and a HUMΛN-operated OpenSSH CA.

Setup

  • Enroll one OpsNode per host (or fleet agent) with an AgentPassport bound to the org.
  • Register only typed remediation actions as real capability nodes (examples: host.service.restart, service reload, certificate rotate) — the privileged helper rejects arbitrary shell input.
  • Install-time Delegations may pre-authorize narrow scopes; moment-of-action HITL still mints a scoped Delegation for each approved proposal.
  • Trust the HUMΛN OpenSSH CA on hosts for short-lived user certificates (no Teleport or Boundary dependency).

Approval flow

  1. Propose — typed action_id + target + parameters (never a free-form command string).
  2. Review — CLI status, Companion Canvas opsgraph_remediation_approval, or MCP get: authority, target, risk, expiry, verification plan, recovery proof.
  3. Approve or deny — approve mints a short-lived scoped Delegation; deny closes the proposal.
  4. Dispatch — OpsNode executes only through the constrained helper after Delegation checks; provenance + Ledger evidence are recorded.
  5. Recover — incident recovery requires product-outcome verification, not merely “restart succeeded.”

Surfaces stay in lockstep: CLI human opsgraph remediation propose|status|approve|deny|dispatch and human opsgraph access certificate; MCP human.operations.remediation.* / human.operations.access.certificate.issue; SDK proposeRemediation, approveRemediation, dispatchRemediation, issueAccessCertificate.

Full guide: docs/guides/opsgraph/remediation.md.

Failure modes

Mode Correct posture
L1 ≠ L3 Edge/host role green does not imply product_capability health
Import ≠ runtime Nagios Nagios import is teacher-only (runtime_monitor_of_record: false); OpsGraph probes signals directly
Analytics not dashboard-only Every module is Companion + MCP + CLI askable
Surface drift Companion tools, MCP human.operations.analytics.* / remediation tools, and human opsgraph verbs share descriptors
Silent canonicalize No auto-promotion; staging never silently labeled production
Heartbeat-only Rejected by defineServiceProfile / Process Contract validation
Restart ≠ recovery Recovery needs consecutive passes + verified product outcome (Phase 2 + Phase 3)
Staging pages like prod Environment-aware alert defaults; override only when configured
Discovery → act Inventory/monitoring never imply remediation authority; Delegation + HITL required
Raw shell remediation Rejected — only registered typed actions via constrained helper
Long-lived SSH keys Human access uses short-lived, revocable HUMΛN CA certificates

Operator runbook: docs/runbooks/opsgraph.md.

Canon-principle connection

  • Human-in-the-Loop — confirmation before canonicalizing Success Profiles; remediation requires explicit HITL approval before dispatch
  • Explicit Uncertainty — proposals carry knowledge_class; insufficient baseline → uncertain, not false healthy; recovery without outcome proof stays unresolved
  • Cryptographic Verifiability — signed provenance and Ledger evidence for every remediation and human-access lifecycle event
  • Capability-First — typed actions are real capability_nodes, not a bespoke shell allowlist
  • Magic by Default — scaffold + family templates; full control via define APIs
  • Infrastructure for Infrastructure — Profile Pack SDK + harness before one-off service knowledge
  • Audience separation (kb/167) — telemetry stores remain platform-consumable
  • Provenance — evaluation history retained (≥30d) for Stability Analytics ranking

References

  • plans/opsgraph_prd.md — product Canon
  • docs/guides/opsgraph/ — setup, monitoring, remediation, profile-sdk, pack-catalog, discovery-handshake, nagios-import, analytics
  • packages/opsgraph-bundle/src/analytics-tools.ts — shared analytics descriptors
  • MCP: OpsGraph MCP tools
  • Getting started: OpsGraph getting started

← All guides