Delivery step 05
Release with operational evidence
A workflow goes live only with its tests, security checks, model evaluations, dashboards, and runbook in place, and with a named owner who answers for it after release.
What happens
- Run the conformance tests, the security review, and the model evaluation against the release candidate.
- Publish the dashboards and the runbook for the workflow.
- Name the owner who answers for the workflow once it is live.
How the platform carries it out
- A release gate blocks any prompt, skill, or model change that does not pass its evaluation.
- Each release is recorded in the audit ledger, with the version of every model and schema it uses.
- Monitoring tracks rejection rate, citation failures, and queue depth. A drop below its threshold returns the workflow to the previous version.
AI role
A judge model scores outputs against a rubric, and its scores are checked against human-reviewed examples. A person signs off the scores.
Approval
The release owner approves go-live. The owner can start a rollback at any time.
Systems involved
- Azure Monitor and dashboards
- Release gate
- Audit ledger
- Runbook library
Reference components
- claims skill release gate
- claims prompt grader suite
- claim audit ledger with hash chain
Pre-sizing
Business case, effort, time, and cost for this step.
These are planning estimates for scoping. They are based on the work listed here and must be checked with the client before a commitment is made.
Business justification
A live workflow without tests, dashboards, and an owner becomes a support cost. Release evidence makes each release repeatable and makes a rollback a standard action rather than an emergency.
Business value
- Each release comes with evidence that it works and is safe to use.
- Problems are seen on a dashboard before a user reports them.
- Any workflow has a named owner and a runbook from the first day.
Documentation to produce
- Test and security plan
- Release gate definition
- Dashboard and alert specification
- Runbook and ownership record
Work breakdown and effort
| Work item | Person-days |
|---|---|
| Write the conformance and security test plan | 5 |
| Build the release gate pipeline | 6 |
| Build the monitoring dashboards and alerts | 5 |
| Write the runbooks | 4 |
| Handover and ownership sign-off | 2 |
| Total effort | 22 |
Delivery time
3 weeks for the full step.
Resources required
- DevOps engineer: 1 FTE
- QA engineer: 1 FTE
- Security reviewer: 0.25 FTE
- Operations owner: 0.25 FTE
MVP: 2 weeks
Scope
- Release gate for the first integration and the first AI task
- Dashboards for queue depth, rejection rate, and citation failures
Exit criteria
- A release candidate passes the gate
- A rollback is rehearsed and recorded
Assumptions
- Azure Monitor and the pipeline tooling are available
- An operations owner is named before go-live
Risks
- Security review finds issues late in the cycle
- Dashboards are built before the measures they show are stable
Cost calculation
Enter your own day rate and weekly platform cost. Until they are entered, the calculator shows effort and delivery time only.
- Effort
- 22 person-days
- Delivery time
- 3 weeks
- Labor cost
- 0
- Platform cost
- 0
- Total estimate
- 0