Pre-release assessment, runtime controls, or both? A security leader's decision guide
They answer different questions, so this is usually not an either/or decision.
Direct answer
They answer different questions, so this is usually not an either/or decision. Pre-release assessment answers "how can this system be broken, and how badly?" before launch. Runtime controls answer "what is this system allowed to do right now?" every time it acts. Our recommendation: treat assessment as the way you learn the failure modes, and runtime controls as the way you enforce limits on them in production.
A practical way to scope each layer
- Every deployment, including read-only and summarization tools, benefits from both layers. A system that only reads and summarizes can still leak sensitive data through its outputs, so runtime policy matters there too - not just logging.
- If the system calls tools, APIs or internal systems, the case for runtime enforcement gets stronger: assessment maps how it can be manipulated (the OWASP Top 10 for LLM Applications is a useful taxonomy - LLM01 is prompt injection, genai.owasp.org/llmrisk/llm01-prompt-injection), and runtime controls bound what manipulated behavior can actually do.
- If it acts autonomously across systems, or touches money, customer data or production infrastructure, we recommend requiring both layers plus human-confirmation gates for irreversible actions.
Questions that settle your decision
- What is the worst action this system could cause if an attacker fully manipulated the model?
The worse the answer, the stronger the case for runtime enforcement.
- How would you know the system was manipulated after launch?
If there is no answer, you need runtime policy enforcement with audit logs, not just monitoring.
- What did pre-release testing actually cover?
A demo and a structured assessment are different things. NIST's AI Risk Management Framework separates these kinds of work into distinct functions - GOVERN, MAP, MEASURE and MANAGE - with evaluation and ongoing risk handling as complementary activities, not substitutes (airc.nist.gov/airmf-resources/airmf/5-sec-core).
Where Fencio fits
Fencio is an AI firewall that checks what an agent is about to do before it acts - evaluating content, context and intent, and enforcing policy outside the model on supported integration paths.
Say yes to the AI your business wants.
Book a 30-minute call. Bring your agents and your rules.