Behind an Inflated Percent, Reality

Beware the glib statistic. Case in point, the claim by the Federal Aviation Administration (FAA) that 94% of the airplanes in the airline fleet are covered, or will be, by Aviation Safety Action Program (ASAP) crew reporting and Flight Operations Quality Assurance (FOQA) data. FOQA data tell what happened in terms of an airplane’s deviation from prescribed procedures; ASAP provides the human element, the explanation for why the event occurred.

The two programs are essential and complementary safety building blocks. As such, one would think they would be required as part of an airline’s FAA-issued operating certificate. They aren’t; why they are not is a separate discussion. For the moment, both are voluntary safety programs.

In its January 2010 report, “Answering the Call to Action on Airline Safety and Pilot Training,” on page 14 of this 200+ page report, we see the 94% figure touted by the FAA (remarks in parentheses are my comments):

“• Twenty-two operators currently have both FOQA and ASAP programs in place (Note: this is 22 of 98 Part 121 scheduled airlines, or slightly fewer than 25%, which is considerably less enthusiastic than the FAA’s 94%).

• Twenty-eight operators have ASAP and state their intention to implement FOQA (Note: not now, but sometime in the future).

• Ten operators with ASAP did not state their intention to implement FOQA. (Note: we can presume they are not).

• Nineteen operators who have neither program at present stated that they will establish one or both. (Note use of weasel words).

• Three operators stated that they do not intend to establish either FOQA or ASAP (Thereby making a mockery of the FAA’s oft-proclaimed “one level of safety” by not planning to have in place precisely those programs needed to routinely evaluate the safety of flight operations. Note also that the responses total up to just 82 of 98 carriers asked to provide responses to the FAA).

“To summarize, the responses show that:

• Ninety-eight percent of aircraft operating under Part 121 are flown by operators that have, or intend to implement, ASAP (Note: this is based on 82 airliners queried, but the percentage under the most generous interpretation is closer to 80% given that there are 98 carriers. Note also that the metric has subtly shifted from operators to the percentage of aircraft).

• Ninety-four percent of aircraft operating under Part 121 are flown by operators that have, or intend to implement, both programs (When will these ASAP/FOQA programs be in place? Not stated; could be in 5 or 10+ years).”

A typical FAA report; hyperbole, not clarity.

A typical FAA report; hyperbole, not clarity.

It’s important to note that this “action” plan was launched after the National Transportation Safety Board (NTSB) last spring identified endemic, industry wide safety deficiencies that were not limited to the circumstances surrounding the fatal crash of Colgan Air flight 3407, a Dash 8-Q400, in February 2009.

On 2 February, the NTSB held its final hearing on the crash investigation, in which some two dozen recommendations were issued on airplane design, crew training, and safety programs needed at carriers across the industry. The FAA’s “Call to Action” report issued on the eve of this hearing, metaphorically speaking, was an attempt to blunt criticism of lax FAA oversight.

It didn’t work. NTSB Member Robert Sumwalt groused, “I don’t know where this 94% came from, and I don’t believe it.”

Colgan Air was a so-called regional carrier, operating short hop flights for Continental Airlines.

Sumwalt said, “Only 2 of 27 regional carriers have FOQA programs.”

That’s a scant 7%.

Sumwalt went on, “Even if it was 98% [instead of 94%], the 2% [not participating] would mean the bottom feeders” who might most benefit from FOQA programs.

NTSB Chairman Deborah Hersman recalled, “Colgan last May said it would have FOQA by last July.”

Still no FOQA at Colgan.

NTSB staffer Roger Cox noted that the Dash 8-Q400 twin-turboprop “has no FOQA data being acquired in the U.S.”

The airplane is operated in Europe, but FOQA data acquired there is not shared here.

The 94% figure pushed by the FAA takes two separate measures – those that already have and those that intend to have ASAP/FOQA – and lumps them together. This creative addition yields a comforting situation that does not apply today and who knows when it will apply in the future?

Better to have started with all airlines subject to FAA oversight, then listed those that have ASAP/FOQA and the number of planes operated by each carrier. Then do the same for those carriers without ASAP/FOQA who plan to implement in the next two years. And, finally, those carriers who are more than 2 years away or not planning ASAP/FOQA programs at all.

From these simple, straightforward numbers, one can tabulate far more meaningful rates of participation. The present tortured but conveniently obscure method reminds one of the aphorism, “You can drink freely from a clear stream and suspect poison in muddy water.”

The FAA has dumped a heap of mud into the public waters. Inevitably, questions would be raised about the credibility of the agency’s rosy 94% estimate.