|Benchmarking Workshop Examines
Your airline has implemented a
voluntary, nonpunitive safety data reporting program, but
how do you know how well it works? Are the data it produces
actually improving safety at your carrier, and if so, how
can you tell? These are just some of the questions Capt.
Leja Noe (Mesa), ALPA Training Council chairman and Central
Air Safety chair for her pilot group, posed to the attendees
at a Tuesday Air Safety Forum workshop titled
“Benchmarking—Good and Bad Practices.”
Noe went around the room,
asking the more than two dozen workshop participants, “What
is the biggest challenge/success with your voluntary safety
program?” Following the ensuing discussion, she facilitated
a brainstorming session to determine what constitutes a
truly effective program, asking attendees to submit written
responses addressing four categories—objective measurement,
feedback and education, FOQA, and networking for
safety—which the group then reviewed.
Attending pilots like
Captains Skip Sampson (PDT) and Greg Downs (UAL) talked
about the strengths and challenges of their airlines’
programs, and ways they are working to make improvements.
Mitch Serber, a former Comair pilot who works with the
Aviation Safety Information Analysis and Sharing (ASIAS)
system for MITRE Corp., emphasized the complications
inherent in developing effective benchmarking tools. He
stressed that programs will often vary depending upon the
dynamics of the group.
“Are you receiving reports
on all related occurrences?” Serber asked the group, noting
that a lack of sufficient data can negatively affect the
quality of research findings.
Dr. Doug Farrow, the FAA’s
acting manager of the Air Carrier Training Systems &
Voluntary Safety Programs Branch Program and manager of the
Advanced Qualification Program (AQP), agreed, adding, “You
need both the numerator and the denominator to clearly
evaluate what’s happening.”
Workshop attendees reviewed
a preliminary gap analysis tool that Noe and other ALPA
safety representatives are developing for evaluating the
effectiveness of safety reporting programs. The tool
includes a checklist of objectives and the processes for
The Forum’s benchmarking
workshop activities complemented exercises and continued
discussions initiated at ALPA’s Safety Data Sharing
Symposium, which the Association presented last February.