Testing drone autonomy in controlled public safety scenarios
Drone autonomy can support searches, perimeter awareness and rapid site assessment, but public safety agencies need evidence before allowing algorithms to influence live decisions. The safest way to test drone autonomy algorithms in a controlled public safety scenario is to progress from simulation to a small, observable live trial.
The trial should answer one operational question, show where human control remains necessary, and produce records that satisfy CASA, privacy and community expectations. It should not become an unmonitored operational deployment under the label of research.
Set the objective and governance boundary
Begin with a narrowly defined task, such as detecting a marked hazard, locating a training target or mapping a safe route through a closed site. State which outputs the software may influence, what triggers a decision, and how it hands control back to a remote pilot. Broad claims about autonomous policing or fully automated rescue create risks that a single trial cannot answer.
Create a small governance group involving the operational owner, pilot, site safety officer, privacy lead and legal adviser. Set pass/fail criteria, prohibited conditions, data rules, stop criteria and an escalation path before entering the test area. The person holding the safety veto should be able to terminate the trial without commercial or political pressure.
Build a controlled and reversible environment
Use layered evidence: software simulation, hardware-in-the-loop testing with flight controls, and then a contained live flight over a benign area. Apply approved geofencing, airspace coordination, physical boundaries and an immediate stop procedure. Simulate faults before testing them physically, and keep the first live trials away from crowds, roads, critical infrastructure and active emergency operations.
Australian conditions should shape the site and test schedule. An urban site around Melbourne can expose the system to buildings and reflective surfaces, Sydney Harbour can test glare and wind, and rural New South Wales can represent vegetation, dust and long sightlines. Test only within the approved risk envelope, record visibility and weather, and treat smoke, haze, rain and low light as separate operating cases rather than one generic scenario.
Exercise the autonomy functions
A credible trial exercises perception, object classification, tracking, route selection, geofence response and communications-loss behaviour. Under the approved test protocol, introduce faults such as delayed observations, unavailable cloud connectivity and low battery states, then verify whether the aircraft returns to a safe state or requests human intervention. Avoid testing over uninvolved people until the lower-risk layers demonstrate reliable detection and containment.
Score detection accuracy, false positives, latency, intervention rate, route efficiency, power use and handover stability against a documented human baseline. Use varied clothing, vehicles, lighting and backgrounds so the dataset does not reward one training environment. Repeat important cases to reveal inconsistent behaviour and preserve less obvious failure modes for later review.
Capture evidence and failure modes
Use independent observers to compare each run with expected outcomes and with the pilot’s judgement. Record near misses, unsafe suggestions, confusing alerts and every manual intervention, not just successful routes. A system that completes the task while creating excessive alerts may still be unsuitable for emergency use, so usability and workload belong in the evaluation.
Store timestamped logs, video, inputs and decision outputs under agreed access and retention rules. If cameras encounter identifiable people, private premises or domestic activities, minimise collection, restrict access and document deletion. Separate operational evaluation from marketing claims, and release only aggregated findings unless the agency has a lawful basis and clear public-interest rationale for more detail.
Meet Australian regulatory and community expectations
In Australia, begin with CASA and confirm whether the activity falls under standard Part 101 operations, an approved category, or the organisation’s Part 107 arrangements. Australian public safety teams commonly refer to these systems as RPAS and require a formal operational risk assessment. Night or beyond-visual-line-of-sight work must be checked against the relevant authorisation and risk controls; an internal policy or supplier’s assurance does not replace regulatory approval.
US agencies can use the Center’s step-by-step night operations guide to understand how a Part 107 waiver is handled, but it has no legal effect in Australia. The comparison should inform questions for CASA, never substitute for the Australian approval pathway or an agency-specific operations manual.
Privacy work should reference the Privacy Act 1988 and the Australian Privacy Principles where applicable, alongside relevant state or territory legislation. The local procurement market often sells aircraft, analytics and fleet-management platforms through separate supplier contracts, so agencies must assign data ownership, security duties and operator responsibilities explicitly. Brief affected communities and local representatives before imaging private property or collecting information beyond the stated public-safety purpose.
Compare evidence and authorise the next stage
The strongest authorisation case combines evidence from each environment rather than relying on a successful flight alone.
| Evidence layer | What it demonstrates | Main limitation | Appropriate decision |
|---|---|---|---|
| Software simulation | Behaviour across repeatable scenarios and injected faults | May not reproduce physical flight dynamics | Revise requirements or algorithms |
| Hardware-in-the-loop test | Integration with controls, sensors and communications | Does not confirm safe movement in the real environment | Prepare for a supervised field trial |
| Contained live trial | Performance under local environmental and operational conditions | Carries higher safety, privacy and cost exposure | Deploy, redesign or discontinue under formal review |
Use the comparison to decide whether a result warrants another simulation, a revised configuration, a supervised field trial or no deployment. Involve the same safety and privacy reviewers at each gate, and keep the final decision with the agency rather than the algorithm vendor.
- Define one operational question and measurable pass/fail criteria.
- Record CASA, airspace, land, privacy and command approvals.
- Progress from simulation to live flight in reversible stages.
- Appoint a site safety officer with immediate stop authority.
- Log interventions, failures and edge cases under agreed data controls.
- Publish a plain-English summary of purpose, oversight, findings and next decision.
Agencies ready to proceed should contact the Center for Unmanned Aircraft Systems in Public Safety, bring the draft scenario to a multi-agency safety review, and obtain written confirmation from CASA and the relevant state authority before the first live flight.