The Impact of Fatigue on Threat and Error Management Performance During Line Operations Safety Audit Observations

By JEANIE HOOD, AMC/SEF, AND DR. STEPHEN POWELL, JENNIFER SAGE, AND BRITTNEY HENDERSHOT, SYNENSYS GLOBAL

Fatigue is defined by the International Civil Aviation Organization as a “… physiological state of reduced mental or physical performance capability resulting from sleep loss or extended wakefulness, circadian phase, or workload (mental and/ or physical activity) that can impair a crew member’s alertness and ability to safely operate an aircraft or perform safety-related duties.” Between 2003-2018, 32 crew fatalities were associated with fatigue across the Air Force, with four fatalities attributed to the Mobility Air Force. Fatigue-related mishaps accounted for over $2.4 billion of medical and material costs. While only 4 percent of all mishaps had fatigue as a factor, these mishaps accounted for over 18 percent of the total cost. More than 25 percent of fatigue-related mishaps were Class A mishaps.

Threat and Error Management (TEM) is the active process of identifying, recognizing, managing, and mitigating threats and errors before they become undesired aircraft states, which lead to mishaps. TEM performance is audited annually by Air Mobility Command (AMC) Safety through the proactive Line Operations Safety Audit (LOSA) process. Threats are events or conditions outside the control of the crew that increase operational complexity. Errors are crew deviations from intentions or expectations. Undesired states are events that degrade the margin of safety—usually the result of a mismanaged error. The LOSA process focuses on the collection of threats, errors, and undesired states during operational missions. AMC has been conducting LOSAs since 2011 and has collected over 3,000 LOSA observations.

In 2018, AMC Safety began proactively collecting crew fatigue data during LOSA observations to better understand the state of fatigue across different aircraft and aircrew. More than 2,400 crew members from AMC voluntarily completed fatigue surveys during LOSA flights. Crew members included pilots, flight engineers, and loadmaster crew members from the C-5; pilots and loadmasters from the C-17; pilots and boom operators from the KC-135; pilots from the C-40 and C-21; and aeromedical evacuation (AE) crew members, including flight nurses and AE technicians.

The LOSA fatigue survey used was adapted from the Samn-Perelli Scale and the Karolinska Sleepiness Scale, which are validated and reliable fatigue and sleepiness survey instruments used by commercial aviation and other high-risk transportation operators. Sleepiness is a condition of current alertness, whereas fatigue is considered a cumulative state of feeling tired. The survey was anonymous, and crews were asked by the LOSA observer to complete the survey prior to the top of descent during a low workload period so it would not interfere with operational duties. This time period for survey completion was selected based on historic LOSA trends that indicate more undesired aircraft states occur during the descent/ approach/landing phase of flight than any other phase of flight. The response rate was very strong at 91 percent. The LOSA observers collected the paper surveys and entered the data, along with all other LOSA observational data, into the AMC LOSA SafeSight software hosted by the LOSA contractor, Synensys Global. All crew fatigue survey results are presented in Figure 1.

Fatigue results ranged from a low of 7 percent of C-21 pilots reporting being fatigued to a high of 46 percent of C-5 loadmasters being fatigued. Only 2 percent of C-21 pilots reported being sleepy at top of descent, whereas 28 percent of AE crews reported being sleepy at top of descent. The blue dotted line represents the average percentage of all crews fatigued (28 percent), and the yellow dotted line represents the average percentage of all crews that reported being sleepy (13 percent). Most of the crews who reported high levels of fatigue and/or sleepiness also had the longest crew duty days (Figure 2).

Our analysis to determine TEM performance differences focused on cockpit crews due to the similarity of duties across aircraft for comparison purposes. Cockpit crews were considered fatigued for the analysis when one or more cockpit crew members indicated they were fatigued during the observed flight. A total of 552 LOSA-observed sorties were included in the analysis (245 cockpit crews, or 44.4 percent, reported they were fatigued, while 307 cockpit crews, 55.6 percent, reported they were not fatigued). All cockpit crews included in the analysis were reported in Figure 1. Threats, errors, and undesired aircraft states are coded as managed or mismanaged by the LOSA observer (e.g., if a crew member is observed successfully managing an aircraft threat 80 percent of the time, then 20 percent of the time aircraft threats are mismanaged).

The most significant threat, error, and Undesired Aircraft State (UAS) performance differences between fatigued and non-fatigued cockpit crews are provided in the following table.

The ultimate goal of TEM is to successfully manage threats before they result in errors which if mismanaged, can lead to UASs, which are precursor events to aircraft mishaps. While most crew members are aware of the negative performance effects of fatigue, they may not be aware of how fatigue directly impacts their TEM performance, which is critical to maintaining safe flight operations. Our data suggests that fatigue most negatively impacts pilot error management performance and some critical UAS categories.

Type of Management Type Fatigued Crews
Mismanaged %
Non-Fatigued Crews
Mismanaged %
Threat Management Cargo/Passenger Compartment 16% 9%
Error Management Manual Handling 49% 43%
Error Management Cross Verification 29% 24%
Error Management Pilot Flying/Pilot Monitoring Duties 19% 14%
UAS Management Unstable Approaches 20% 9%
UAS Management Vertical Deviations 21% 14%
Threat, Error, and UAS Management Performance Differences for Fatigued versus Non-Fatigued
Cockpit Crew Members

Long haul (C-5/C-17) crews and AE crews reported the highest level of fatigue across the LOSA sample. These crews also experienced the longest duty days. Implementing TEM strategies to address fatigued crews’ ability to successfully manage all threat and error types (especially those that crews are most vulnerable to mismanaging) is essential to avoiding and mitigating UASs that can negatively impact flight safety.

Future LOSA data collection will continue to include fatigue surveys, Aviation Operational Risk Management fatigue scoring, and other fatigue-related narrative data to develop actionable information for leaders, crews, and teams to use to optimize TEM performance even when fatigued.