PHASE: Process-oriented Hazard Analysis for AI Systems

From Silos to Systems: Process-Oriented Hazard Analysis for AI Systems

Authors:

Shalaleh Rismani, Postdoctoral Researcher, McGill University, Montreal, Canada

Roel Dobbe, Assistant Professor, Delft University of Technology, Delft, The Netherlands

AJung Moon, Associate Professor, McGill University, Montreal, Canada

PHASE adapts System Theoretic Process Analysis (STPA), a system safety hazard analysis method developed by Nancy Leveson and John Thomas, for AI systems. It supports system-level hazard analysis by helping analysts examine not only models and datasets, but also the broader development and operation processes in which AI systems are situated.

---

Project Overview

Current AI risk and harm analysis approaches often focus on individual components of an AI system, such as training data or models, in isolation. PHASE takes a system-level view by examining how hazards can emerge from interactions between components, development processes, organizational decisions, users, and deployment contexts.

This project includes:

• the paper, From Silos to Systems: Process-Oriented Hazard Analysis for AI Systems
• the PHASE process for adapting STPA to AI systems
• three case studies involving:
  • linear regression
  • reinforcement learning
  • transformer-based generative models
• templates for conducting PHASE analyses

---

The Three Case Studies

---

Citation

Please cite the paper if you use PHASE in your work.

See the Citation page for BibTeX.