Demo: red-team regression suite in CI with a signed report

Customers can run a packaged adversarial regression suite (single-step and multi-step chains) against their own configuration; blocked attacks provably cause zero unauthorized side effects, and results are signed. This demo runs the suite in a CI job, asserts zero unauthorized side effects, and archives the signed report.

Safe-by-default: local recording targets, no destructive tests, dry-run. No production credentials.

Readiness

• Customer-runnable red-team harness + signed reports: PILOT_READY (customer-pilot ready at supported boundaries).
• Python; embedded(service) and sidecar modes.

Setup

# demo_red_team_ci.py — synthetic fixtures only
from core.eve_sdk import EVE
from core.redteam import RedTeamHarness

IDENT = {"tenant_id": "acme", "principal_id": "agent-1", "session_id": "sess-1"}
eve = EVE(policy="lending_v1", mode="embedded")

# Harness runs against YOUR config with safe local recording targets (no real side effects).
harness = RedTeamHarness(eve, identity=IDENT, dry_run=True)

Code

report = harness.run(suites=["single_step", "multi_step_chains"])

# Every blocked attack must have caused zero unauthorized side effects.
assert report.unauthorized_side_effects == 0, "no attack may cause an unauthorized side effect"
print("cases:", report.total, "passed:", report.passed,
      "unauthorized_side_effects:", report.unauthorized_side_effects)

# Archive the signed report as a CI artifact.
report.write("redteam_report.json")   # signed evidence file for the build

CI wiring (fail the build on any unauthorized side effect):

# .github/workflows/red-team.yml (illustrative)
- name: EVE red-team regression
  run: python demo_red_team_ci.py
# Non-zero exit if unauthorized_side_effects > 0 or any expected-block case executed.

Expected decision / result

Expected evidence

A signed red-team report:

{"report": "redteam", "total": 18, "passed": 18,
 "unauthorized_side_effects": 0,
 "standards_mappings": ["...descriptive..."],
 "certificate": {"signature": "ed25519-...", "canon": "jcs-1"}}

Standards mappings are descriptive, not certification.

Verification command

from core.eve_sdk import verify_evidence
import json
env = json.load(open("redteam_report.json"))
v = verify_evidence("redteam_report", env["certificate"], expected_tenant="acme")
assert v["valid"], v.get("reason")
print("verify:", v["valid"])   # -> True

Zero-side-effect assertion

report.unauthorized_side_effects == 0. The harness records attempted side effects to local targets; a blocked attack must leave those targets empty. Any expected-block case that executed would increment the counter and fail the build.

Cleanup

import os
os.remove("redteam_report.json")   # remove the local CI artifact if not archiving

The suite runs dry-run against recording targets; nothing external was modified.

Limitations

• Safe by default (local recording targets, no destructive tests, dry-run). The suite is representative, not exhaustive; customers add their own fixtures.
• Standards mappings are descriptive, not certification. A green run does not certify compliance.
• PILOT_READY, not production.