Published: 2025

Clever Hans in the Loop? A Critical Examination of ChatGPT in a Human-in-the-Loop Framework for Machinery Functional Safety Risk Analysis

CATEGORIES

RISK-BASED PROCESS SAFETY ELEMENTS

Research Summary

This feature paper evaluates ChatGPT within a human‑in‑the‑loop (HITL) framework for machinery functional safety risk analysis under ISO 12100. It shows that prompt design strongly affects accuracy and usability: longer initial prompts improve correctness, while shorter iterative prompts maintain efficiency. The study reports case‑study evidence that structured prompting plus expert oversight can achieve high agreement with ground truth risk analyses. For PSM, the work is a concrete template for using LLMs in risk assessments (e.g., HAZOP, LOPA) where prompt patterns, iteration strategies, and HITL controls are critical to safe deployment. It also highlights hallucination risks and the need for explainability and governance, directly connecting prompt engineering to responsible AI in safety‑critical environments.

AUTHORS

Padma Iyenghar

CITATIONS

P. Iyenghar, “Clever Hans in the Loop? A Critical Examination of ChatGPT in a Human-in-the-Loop Framework for Machinery Functional Safety Risk Analysis,” Eng, vol. 6, no. 2, art. 31, Feb. 2025, doi: 10.3390/eng6020031.