C16
Layer 4 - Orchestration & Ecosystem
Continuous Adversarial Validation and High-Assurance Verification
Continuous red-team and regression suite for adversarial inputs. Production gates run a baseline corpus before deployment; failures block release.
Why
- Attackers adapt. Also, some invariants must be provably true.
What
- Two complementary practices: Continuous adversarial validation: CI/CD harness for injection, tool misuse, poisoning, and replay regressions
- Formal verification (where needed): verifier that checks planned actions against invariants for high-impact tools
How
- maintain an abuse case library and regression corpus
- gate deployments on exploit success thresholds
- verifier service sits between plan and execution for high-impact tiers
- counterexamples become replayable incident tests
Evidence
- exploit success trend and pass/fail gates
- verifier coverage (% high-impact actions checked)
- counterexample catalog and mitigation tracking
Failure modes
- one-off red team with no regression suite
- tests that don’t match production tool surfaces
- verification performed “on paper” rather than in the execution path
NIST AI RMF alignment
C16 maps to MEASURE and MANAGE. See the framework paper for the specific subcontrol mappings.
ISO/IEC alignment
C16 maps to ISO/IEC 27001. Typical evidence: see the Evidence section above.