Ask what the system actually does
The label AI can describe very different functions. A system might estimate counts from images, forecast arrival patterns, propose scenario combinations or generate a written summary. A pedestrian simulation may use entirely different methods. Identify each component and its role before evaluating a broad claim that the system “optimises crowd safety”. Such language is not a testable description of its performance.
Write a purpose statement with a defined decision, location, operating phase and intended user. Then document exclusions. A model designed to compare routine arrivals may say little about emergency behaviour, people needing assistance or the effect of an unplanned route closure. A clear exclusion is more useful than an implied capability that no one has examined.
Require traceable inputs and outputs. A reviewer should be able to identify the site revision, attendance assumptions, route-choice rules, software version and scenario settings behind a result. If an AI-generated narrative introduces a fact that is absent from the evidence pack, treat it as an unsupported claim until checked—not as a useful completion of missing information.
Separate verification from validation
Verification examines whether the implementation matches the intended model. This includes checking geometry, connections, units, boundary conditions and data transformations. A path accidentally connected through a fence can produce convincing-looking movement while representing an impossible journey. Small test cases and inspection of intermediate outputs can expose errors that remain difficult to see in a full animation.
Validation asks whether the model is sufficiently representative for its stated use. That may involve comparison with relevant observations, examination of known limitations and independent challenge. Evidence from a different venue or audience needs a reasoned argument for transfer. Validation is not a universal badge that makes every later application appropriate.
Keep calibration distinct from an independent check where possible. If the same observations were used to tune the model and demonstrate its agreement with reality, say so. Explain measurement uncertainty and the aspects of behaviour that the comparison does not cover. A close match to one aggregate total may conceal incorrect local patterns or timing.
Compare scenarios with a common ledger
Begin with a documented reference case, not an assumed prediction of what will happen. Change inputs for a reason, record the rationale and retain the same comparison measures where they remain appropriate. For stochastic models, review variation between runs rather than selecting a favourable animation. Where the method is deterministic, explain the resulting limitations instead of implying that repeatability proves accuracy.
| Scenario | Evidence question | Operational review |
|---|---|---|
| Reference arrangement | Are layout, timing and behaviour assumptions traceable? | Confirm the study matches the current plan. |
| Changed arrival pattern | What supports the timing range? | Review queue space and partner interfaces. |
| Reduced route availability | Is the alternative genuinely usable? | Check receiving space, staff and authority. |
| Different route preferences | How sensitive is the result to attendee choice? | Review wayfinding and observation needs. |
Do not use comparison to conceal an unacceptable reference assumption. If every scenario depends on an unverified opening or unavailable staff resource, the appropriate next step is to resolve that dependency. A result can be informative while remaining insufficient for an operational decision.
Keep the challenge questions open
Which input changes the conclusion most?
Identify material sensitivities and whether plausible input variation changes the decision. Prioritise better evidence where it can meaningfully reduce uncertainty.
What observation would contradict the planning case?
Define the observation, its coverage and reporting route. Avoid relying on a dashboard value without knowing its delay, uncertainty and blind spots.
Can the proposed response actually be delivered?
Check authority, staffing, equipment and coordination. A response described in software is not available merely because the interface offers it.
These questions should be discussed with the relevant operational specialists. The modeller may understand the software while another person understands the site, accessibility or transport interface. A structured review brings those perspectives together without allowing any single tool to stand in for the whole planning process.
Return the decision to named people
Record the decision owner, reviewers, evidence considered, limitations accepted and issues still open. Distinguish review of the modelling method from approval of the event arrangements. The people authorised to accept operational risk or approve a change are determined by the event’s governance and applicable requirements, not by the software supplier.
Changes after review need an impact check. Moving a concession, altering a programme or closing a path may invalidate an assumption that appeared minor. Maintain version links between the approved arrangements and the study. After the event, compare observations with the planning case without rewriting the original assumptions to make the forecast appear better.
Useful wider references include the NIST AI Risk Management Framework, ISO 31000 risk management guidance and ISO 22320 incident management guidance. These support governance thinking; citing them does not establish compliance, certify a model or approve a festival. Event-specific guidance from HSE and the Purple Guide should be read within its jurisdiction and scope.