So, What Can I Do With MFOQA Analysis?

During a recent Safety Investigation Board (SIB), the Military Flight Operations Quality Assurance (MFOQA) Flight Data Analyst was asked, “What could MFOQA analysis provide to assist the investigation of a mishap?” Although this is not the first time the Ops Risk Assessment and Management System (RAMS) team has received this question, it highlighted the need to periodically refresh the aviation community, both crews and leadership, of the capabilities MFOQA analysis provides. Therefore, it is essential to review the uses of MFOQA analysis and associated restrictions and how crews can use analysis products as a preflight tool to improve situational awareness. The analysis can also provide insight into how SIB Investigation Officers can acquire in-depth aggregate information to determine if the mishap crew was operating within the current norms and how leadership can measure the risk exposure to their crews.

The two primary points to remember about MFOQA are—

Point 1: The only limit to what MFOQA analysis can provide is your imagination of what you want to look at and whether your aircraft records the parameters required to define the event. (Adding new parameters to your Mission Design Series [MDS] is a story for another day.)

Point 2: The MFOQA Task Force that established the framework that the MFOQA program operates under firmly believed there were sufficient evaluation programs in place to ensure the crew force was proficient and ready to execute the National Defense Strategy.¹ Therefore, there was no requirement to use MFOQA analyses to monitor an individual Airman’s performance. Instead, the task force wanted a tool that could change anecdotal beliefs into statistical evidence highlighting what was happening in the field on an aggregate level. To be more precise, the overarching goal of the MFOQA program is to identify mishap precursors to allow leadership to develop mitigation options. To support their firm belief in crew proficiency, the task force created restrictions for the use of MFOQA analysis in both Department of Defense (DoD) and U.S. Air Force guidance. Although wording has changed and instructions renamed, the restrictions still follow the original intent of the task force. DoD Instruction (DoDI) 6055.19 states, “Data collected for, or analysis generated from, AHIRAPs [Aviation Hazard Identification and Risk Assessment Programs] must not be used to: (a) Monitor personnel performance to initiate qualification downgrade or decertification.”² Air Force Instruction (AFI) 91-225, Aviation Safety Programs, further clarifies that “Data collected for or analyses generated from aviation safety programs shall not be used to initiate crew qualification downgrade, take adverse personnel action, or monitor personnel performance.”³ In addition, the AFI expounds on the definition of adverse actions.

From inception, the goal of the MFOQA program was to look at flight operations from the aggregate perspective. Figure 1 highlights how MFOQA analysis can establish a baseline for how the community performs a specific task, and Figure 2 shows trends in unstable approaches. To accomplish this analysis, the MFOQA software relies on hundreds of points of interest (POIs), called Routine Operational Measurements. The MFOQA software also supports warning events (e.g., flap or MDS overspeed) and action points (e.g., configuration changes or touchdown points) to alert the team of unsafe situations. With your feedback and a little time, the MFOQA Software Development Team can add almost any POI analysis you may want to examine. Due to limits imposed by the parameters recorded by each aircraft, not all requests can be supported, but you will not know if you do not ask.

The following graphs are examples of how MFOQA analysis can be beneficial in showing statistical trends.

In addition, MFOQA analysis can measure the impact of seasonal changes and environmental influences at a specific location, such as the impact of spring thunderstorms on stable approaches (Figure 2) and the ground proximity warning system “Terrain Fly UP” warning received while on glideslope flying into Runway 23 at Spangdahlem Air Base, Germany (Figures 3 and 4).

Figure 5, on the other hand, shows how leadership can evaluate the success of mitigation efforts based on MFOQA analysis over an extended period.

Early in the life of the MFOQA program, it became apparent that the software developed for aggregate analysis could also examine a singular event (Figure 6), target a specific event (Figure 7), or show ground track and event location by overlaying data on a map (Figure 8).

These statistics lead to an issue that is near and dear to me—two-way communication. It is imperative! MFOQA analysis only tells us what happened; it does not tell us why it happened. There are times when the analysts need to contact the aircrew to add context to an event. For example, the MFOQA software alerts the analysts when a KC-135 aircrew attempts to take off with the speedbrake handle out of the detent. After several of these events occurred, the analysts then contacted one of the involved aircrews. Interestingly, the aircrew stated that they were convinced the speedbrake handle was not out of the detent. Maintenance checked the sensor and found it to be defective. Therefore, this event was removed from the analysis database to prevent skewing aggregate analysis.

Crew protection from monitoring and initiating adverse personnel action is spelled out in DoDIs and AFIs. In addition, the MFOQA software is designed to look at variable POIs to provide a graphic that can establish an operational baseline and support the investigation of specific locations during mission planning. The software also enables safety investigation teams to examine both aggregate and singular mission data sets to garner a better understanding of what might have caused a mishap and provides commanders at all levels with the capability to assess mission risk and crew proficiency. The MFOQA software allows commanders to measure mission creep to determine if crews are getting too comfortable with the risk associated with their mission and evaluate the success of mitigation efforts. Finally, two-way communication is essential to improving the analysis and providing the MFOQA analysts with issues that are important to the field.

Until we can meet in person, Fly Safe!

1 https://www.defense.gov/Explore/Spotlight/ National-Defense-Strategy/
2 DODI 6055.19 Aviation Hazard Identification and Risk Assessment Programs (AHIRAPs), April 11, 2017, 3.2.e(2)(a) 3 AFI 91-225, Sep 2, 2019, Safety Programs. 1.2.3.3.