Development of a Holistic Assessment Framework for the Design of AI-Based Automation

Abstract

There is a need to ensure that the application of artificial intelligence (AI) in increasingly automated operations is safe, human-centric, and trustworthy, and respects ethical principles. To this end, this paper presents an innovative holistic assessment framework to support certification-aware design of AI-based sociotechnical systems with a range of levels of automation along multiple design stages from low to high technology and human readiness levels (TRLs/HRLs). The holistic scope considers a range of relevant key performance areas (KPAs): safety, resilience, security, Human Factors, accountability, responsibility, liability, efficiency, societal sustainability, and environmental sustainability. The core of the framework is a seven-step cycle that assesses the KPAs for critical scenarios and evaluates the combined performance, including uncertainty and trade-offs. This provides feedback to either adapt the design at the same TRL/HRL or refine it at higher TRLs/HRLs. The framework enacted by a toolbox of assessment methods for the KPAs. The framework has been developed in the aviation domain, but it is formulated in a generic manner, enabling application to various AI techniques and operational domains. Its application is illustrated in detail for an air traffic management use case that employs an AI-based system to support air traffic controllers in sequencing aircraft. It is concluded that the framework provides a viable approach for holistic assessment of AI-based sociotechnical systems.