OSES USE CASE SUMMARY

OSES-AI: An Artificial Intelligence Enhanced Organizational Safety Effectiveness Survey for Aviation and Other Safety-Critical Industries

Executive Overview

Major accidents in aviation, aerospace, nuclear power, and oil and gas operations rarely result from a single operator error. Research spanning more than four decades demonstrates that catastrophic events are often preceded by identifiable organizational warning signs, latent failures, degraded safety culture, leadership blind spots, communication breakdowns, normalization of deviance from safety practices, and ineffective risk management processes.

The Organizational Safety Effectiveness Survey (OSES), originally developed by Dr. Anthony Ciavarelli and Human Factors Associates (HFA), was specifically designed to identify these organizational accident precursors before they contribute to a serious mishap. OSES has successfully supported safety assessments within military aviation, commercial aviation, healthcare, and other high-risk industries by providing leadership with actionable information regarding safety climate, organizational risk, and safety management effectiveness.

OSES is administered online and provides immediate feedback to organization leaders by providing diagnostic statistical charts of key statistical trends of survey questionnaire results.

HFA has applied OSES to assess safety climate and to evaluate implementation of safety management systems for some notable clients, including Qantas Airlines, Bristow Group Offshore helicopter transport, Reach Air Medical transport, the Italian Air Force, and elements of the OSES questionnaire have been used in studies of hospital patient safety programs.

HFA employed OSES over a nine-year period in support of NASA Aviation safety management. NASA sponsored OSES psychometric testing and revision of the questionnaire content to enhance online survey administration and analysis.

The next-generation OSES-AI expands this proven capability by integrating Agentic Artificial Intelligence to provide automated risk identification, predictive analysis, interactive user guidance, statistical interpretation of employee survey results, and executive-level reporting.

The Safety Problem

Investigations of major disasters consistently reveal organizational warning signs that were visible before the accident occurred.

Examples include:

Space Shuttle Challenger (1986)

• Engineers repeatedly expressed concerns regarding O-ring performance.
• Management normalized increasing levels of risk.
• Communication barriers prevented critical technical concerns from influencing launch decisions.

Space Shuttle Columbia (2003)

• Repeated foam strike incidents became accepted as normal.
• Organizational assumptions overrode engineering concerns.
• Safety review processes failed to challenge existing beliefs.

Air France Flight 447 (2009)

• Training, procedural understanding, automation management, and crew coordination deficiencies contributed to the accident sequence.
• Opportunities existed to strengthen operational preparedness and organizational learning.

Boeing 737 MAX

• Production pressures and business objectives influenced safety-related decisions.
• Communication gaps existed between engineering, management, regulators, and operators.
• Safety concerns were not fully elevated or addressed before accidents occurred.

Fukushima Daiichi Nuclear Accident

• Known vulnerabilities to extreme external events were underestimated.
• Risk assessments did not adequately account for low-probability, high-consequence scenarios.
• Organizational preparedness was insufficient for severe contingencies.

Deepwater Horizon / Macondo Blowout

• Warning indicators and operational anomalies were present before the event.
• Production priorities competed with safety considerations.
• Multiple organizational defenses failed simultaneously.

Across these accidents, investigators repeatedly identified organizational conditions that, if recognized and corrected earlier, may have prevented escalation into catastrophe.

OSES-AI Solution

OSES-AI functions as an Organizational Early Warning System. 

The system anonymously gathers workforce perceptions regarding:

• Leadership commitment to safety
• Reporting culture
• Safety communication effectiveness
• Procedural compliance
• Production pressure
• Operational discipline
• Risk management effectiveness
• Learning from incidents
• Workforce trust and engagement
• Hazard identification and mitigation
• Training adequacy
• Organizational resilience

Questionnaire survey results provided to management and safety personnel serve as important feedback used to evaluate to both favorable and unfavorable results. OSES enables a unique and effective means of communication between front line workers and supervisory staff regarding the status organizational climate, safety culture and operational safety performance.

Unlike traditional surveys, OSES-AI uses Agentic Artificial Intelligence to continuously evaluate survey results against established safety science models, accident investigation findings, and High Reliability Organization (HRO) principles.

How Artificial Intelligence Adds Value

1. Automated Detection of Accident Precursors

AI analyzes survey responses to identify patterns associated with:

• Normalization of deviance
• Excessive production pressure
• Procedural and regulatory compliance
• Weak safety reporting cultures
• Leadership credibility problems
• Training deficiencies
• Communication breakdowns
• Organizational complacency
• Inadequate corrective action processes

The system highlights risk areas before they become operational failures.

2. Benchmarking Against Historical Accident Patterns

AI compares organizational survey results with factors identified in major accident investigations.

For example, the system can detect conditions resembling:

• Automation induced accidents, like Air France 447, in part                                                       due to pilot training deficit
• Challenger-style management communication failures
• Columbia-style normalization of deviance
• Boeing 737 MAX decision-making pressures
• Deepwater Horizon production-versus-safety conflicts
• Fukushima preparedness deficiencies

This allows organizations to recognize emerging vulnerabilities before they contribute to an accident chain.

3. Interactive Safety Intelligence Assistant

A built-in AI Safety Advisor provides leaders with:

• Natural language explanations of survey findings
• Risk interpretation assistance
• Statistical result explanations
• Recommended mitigation strategies
• Suggested corrective action priorities
• Follow-up investigation guidance

Executives and safety managers can simply ask:

“What are our highest risk areas?”

“Which findings require immediate attention?”

“How do our results compare to high reliability organizations?”

The AI provides evidence-based answers supported by safety science research.

4. Automated Reporting and Decision Support

OSES-AI automatically generates:

• Executive summaries
• Safety climate dashboards
• Risk heat maps
• Trend analyses
• Organizational resilience indicators
• Mitigation recommendations
• Board-level briefing materials

Reports are customized for executives, safety officers, managers, regulators, and operational personnel.

Primary Aviation Use Case

A commercial airline administers OSES-AI annually across pilots, maintenance personnel, dispatchers, cabin crews, and managers.

The AI identifies a developing pattern involving:

• Increased production pressure
• Reduced willingness to report concerns
• Perceived inconsistency in management response
• Declining confidence in corrective action processes

Although no accidents have occurred, the AI risk engine identifies a combination of organizational factors historically associated with elevated accident potential.

Leadership receives:

• Immediate risk alerts
• Root-cause analysis
• Benchmark comparisons
• Recommended interventions

Corrective actions are implemented before operational failures emerge.

The result is a proactive approach to accident prevention rather than reactive investigation after a loss event.

Strategic Value

OSES-AI transforms safety management from a compliance-oriented activity into a predictive organizational intelligence capability.

The system helps organizations:

• Identify accident precursors before accidents occur
• Strengthen safety climate and culture
• Improve leadership decision making
• Detect emerging organizational risks
• Enhance workforce trust and reporting
• Support Safety Management Systems (SMS)
• Improve operational resilience
• Move toward High Reliability Organization performance

Conclusion

For more than twenty years, OSES demonstrated that organizational safety risks can be measured before they appear in accident statistics. The integration of Agentic Artificial Intelligence now enables OSES-AI to move beyond measurement and become a predictive safety intelligence platform.

By combining proven safety climate science, High Reliability Organization principles, and advanced AI analytics, OSES-AI provides aviation organizations with an unprecedented capability to detect organizational accident precursors, prioritize risk mitigation, and strengthen the safety culture needed to prevent future catastrophes.

REFERENCES AND SOURCE LINKS

Aviation and Aerospace Accident Reports

1. BEA (2009). Air France Accident Report June 2009 Airbus A330-203 BEA Accident Report (published July 204. Final Committee Report.

2.  US Congress (2020, September). The Design, Development, and Certification of the Boeing 737 Max. House Committee on Transportation and Infrastructure.

3.  US Congress (1986). Investigation of the Challenger Accident. Report of the Committee on Science and Technology. House of Representatives 99^th Congress 2^nd Session, October 29, 1986.

4. Columbia Accident Investigation Board. Report, Volume 1, August 2003. Government Printing Office, Washington, D.C.

Nuclear, Oil and Gas Accident Reports

1.  NEA/OECD (2013). The Fukushima Daiichi Nuclear Power Plant Accident. OECD/NEA Nuclear Safety Response and Lessons Learnt. NEA. OECD NEA No. 7161.

2. U.S. Safety – Hazard Investigation Board. (March 2007). Refinery Explosion and Fire: Investigatin Report (Report No. 2005-04-1-TX)

2.Deepwater Horizon Study Group (March 1, 2011). Final report on the investigation of the Macondo Well Blowout. Center for Catastrophic Risk Management, UC Berkeley.

3. Deep Water. The Gulf Oil Disaster and Future of Offshore Drilling. Report to the President. July 2011.  National Commission on the BP Deepwater Horizon Oil Spill and Offshore Drilling.

Challenger Shuttle Report

https://spaceflight.nasa.gov/outreach/SignificantIncidents/assets/rogers_commission_report.pdf

Columbia Shuttle Report

https://www.nasa.gov/pdf/298870main_SP-2008-565.pdf

Air France 447

https://www.bea.aero/docspa/2009/f-cp090601.en/pdf/f-cp090601.en.pdf

Three Mile Island Nuclear Accident

https://en.wikipedia.org/wiki/Three_Mile_Island_accident

High Reliability (HRO) Research References

AHRQ (2010). Becoming a high reliability organization: Operational Advice for Hospital leaders. Prepared by the Lewin Group, Falls Church, VA.

Ciavarelli, A.P. (2008, February). Culture Counts: How does your organization measure up? Aerospace Safety Magazine.  Washington DC: Flight Safety Foundation. 

Ciavarelli, A.P (2007, October). Assessing safety climate and organizational risk. Baltimore, MD, HFES 51^stAnnual Meeting.

Ciavarelli, A., Figlock, R., Sengupta, K., & Roberts, K. (2001). Assessing organizational safety risk using safety survey methods. The 11thInternational symposium on Aviation Psychology, Columbus, OH. 

Desai, V.M., Roberts, K.R., and Ciavarelli, A.P. (2006, winter). Defensive attributions in the formation of perceived safety climate. Human Factors and Ergonomics, 48. (4) 639-650.

Flin, R., Mearns, K., O’Connor, P., and Bryden, R., (2000) Measuring safety climate: Identifying the common features. Safety Science, 34, 177-92.

Gaba, D.M., Singer, S.J., Sinaiko, A.D., Bowen, J.D., & Ciavarelli, A.P. (2003). Differences in safety climate between hospital personnel and Naval Aviators. Human Factors and Ergonomics, 45, 173-185.

Higginbotham, A. (2024) Challenger: A true story of heroism and disaster on the edge of space. NY: Avid Reader Press.

Hopkins, A. (2025). Boeing: The 737 MAX crisis and aviation safety. The perils of profit – driven engineering. CRC Press.

Mearns, Whitaker, S.M., & Flin, R. (2003). Safety climate, safety management practice and safety performance in offshore environments. Safety Science, 41, 641-680. 

Perrow. P (1984). Normal Accidents: Living with high-risk technologies. Princeton University Press.

Reason, J. (1997). Managing the risks of organizational accidents. Brookfield: Ashgate. 

Roberts, K.H. (1990, summer). Managing high-reliability organizations. California Management Review. 32, (4), 101- 113. 

Roberts, K. H. (1993). Culture characteristics of reliability enhancing organizations. Journal of Managerial Issues, 5, 165-181. 

Roberts, K.H. and Bea, R. (2001). Must accidents happen? Lessons from high-reliability organizations. Academy of Management Executive, Vol 5, No.3,70-79.

Robinson, P. (2022). Flying Blind: The 737 MAX tragedy and the fall of Boeing. NY Anchor Books.

Schein, E.H. (1996). Culture: The missing concept in organizational studies. Administrative Science Quarterly, 41, 229-240. 

Singer, J. S., Rosen, A., Zhao, S., Ciavarelli, A.P. (2010). Comparing safety climate in naval aviation and hospitals: Implications for improving patient safety. Health Care Management Review, 35, (2), 134-146.

Turner, B.A. (1978). Man-made disasters. NY: Crane-Russak & Co. 

Vaughan, D. (2016). The Challenger launch decision: Risky technology, culture and deviance at NASA. 

Weick, K.E. (1999). Organizing for high reliability: Processes of collective mindfulness. Research in Organizational Behavior, 21, 81-123.

Weick, K.E. (1987). Organizational culture as a source of high reliability. California Management Review, 29, (2) 112. 

Zohar, D (2002). Safety climate: Conceptual and measurement issues. In J. Quick & L. Tetrick (Eds.), Handbook of occupational psychology (pp. 123- 142). Washington DC: APA. 

Zohar, D. (1980). Safety climate in industrial organizations: Theoretical and implications. Journal of Applied Psychology, (1), 96-102.