US airline service quality survey rises. But what’s behind the numbers?

The annual Airline Quality Rating (AQR) report has been released by Wichita State University, an annual assessment that dates back to 1991 and measures airline performance based on a number of statistical measures. The good news is that in all but one of the categories measured, there was improvement, with only Denied Boardings registering a negative index. Other attributes measured include On Time Performance, Mishandled Bags and the generic catch-all Customer Complaints category that deals with a multitude of sins and slights.

While these measures are certainly germane to travelers—arriving hours late without a bag will definitely sour one’s mood—the weakness of such studies is that the overall experience, with its indignities and annoyances, is very difficult to rank. With new and burgeoning charges, as well as a widespread perception that for many travelers “service” now means little more than a canned greeting and a request for the credit card to pay for amenities, most airlines are still not “delighting” their patrons.

The complete report, covering many pages of methodology, statistics and analysis is available at http://www.aqr.aero/.

No change at the top

The overall rankings changed very little from 2008 to 2009, as the top four airlines remained the same. The top four airlines this year were the top four from last year – Hawaiian, AirTran, JetBlue and Northwest. However, Southwest moved up a notch coming in fifth rather than sixth. Continental, which continually ranks high with such consumer ratings services as JD Powers and Associates moved up two slots to sixth as did US Airways which moved from tenth to eighth place. Frontier retained is seventh-place ranking as did American at ninth place.

ExpressJet ranked tenth, but was not ranked in the year-ago report. Regionals have only been ranked since 2003. In addition, United dropped from 11th to 13th, while Mesa rose from 14th to 12th. SkyWest, displaced Mesa as number 14, after coming in 13th last year. Delta dropped three points to 15th in the survey, while Delta Connections Comair dropped a point to 16th, while and Atlantic Southeast (ASA) retained its 17th place ranking. Finally, American Eagle dropped from 16th to 18th place.

AQR 2010 Statistical Rankings (2009)

1.     Hawaiian (1)

2.     AirTran (2)

3.     JetBlue (3)

4.     Northwest (4)

5.     Southwest (6) 

6.     Continental (8)

7.     Frontier (7)

8.     US Airways (10)

9.     American (9)

10.   ExpressJet (not ranked in 2009 report)

11.   Alaska (5)

12.   Mesa (14)

13.   United (11)

14.   SkyWest (13)

15.   Delta (12)

16.   Comair (15)

17.   Atlantic Southeast (17)

18.   American Eagle (16)

Hawaiian Airlines had the best on-time performance (92.1 percent) for 2009, and Atlantic Southeast had the worst (71.2 percent). Fourteen airlines improved their on-time arrival   performance in 2009. Only six of the 18 airlines rated had an on-time arrival percentage over 80. On-time for 2009 by the industry was 79.4 percent compared to 76 percent in 2008.

AirTran had the best baggage handling rate (1.67 mishandled bags per 1,000 passengers) of all airlines, and Atlantic Southeast had the worst baggage handling rate (7.87 mishandled bags per 1,000 passengers) of all the airlines.

Mishandled baggage was the most consistent area of performance improvement in 2009. All 18 airlines improved their mishandled baggage performance for the year. The rate for the industry decreased from 5.19 per 1,000 passengers in 2008 to 3.88 in 2009.

Southwest again had the lowest consumer complaint rate (0.21 per 100,000 passengers) of all airlines. Delta had the highest consumer complaint rate (1.96 per 100,000 passengers) of all airlines rated.

Customer complaints per 100,000 passengers decreased from 1.15 in 2008 to 0.97 in 2009. The majority of complaints were for flight problems (23.8 percent), baggage (18.7 percent), reservations, ticketing and boarding (15.1 percent), and customer service (13.9 percent).

Second consecutive year of improvements

Considering that 2008 was a down year along with 2009, it is thus no surprise the rankings improved. The AQR rate 18 carriers, including regional airlines which only joined the rankings in 2007, and reported that of the 17 rated in both 2008 and 2009, all but Alaska improved Airline Quality Ratings for 2009. Indeed, Alaska dropped from fifth place in the 2009 rankings to 11th place in 2010.

The AQR reported 2009 marked the second consecutive year airline performance has improved; good news since it wasn’t long ago airlines ranked below the Internal Revenue Service. It was the third best overall score in the 19 years researchers have tracked the performance of airlines.

Having improved their ratings from on-time performance, baggage handling and customer complaints, the only measure that slipped was denied boardings. "You would expect denied boardings to increase as you tighten up on the number of seats that are available," said Associate Professor of Marketing at the W. Frank Barton School of Business at Wichita State University Dean Headley. "When you look at the past 10 years, you find that the airline industry performs most efficiently when the system isn't stressed by high passenger volume. Every time there are more planes in the sky and more people flying, airline performance suffers. Airlines are focused on generating revenues, not necessarily on customer service.”

JetBlue had the lowest involuntary denied boardings rate at 0.00 per 10,000 passengers. American Eagle had the highest involuntary denied boardings rate at 3.76 per 10,000 passengers. Overall, nine airlines improved their denied boardings rate in 2009. Atlantic Southeast recorded the largest improvement, and Alaska had the largest increase. JetBlue and Hawaiian are clearly the industry leaders in avoiding denied boarding incidents. Industry performance was worse in 2009 (1.19 per 10,000 passengers) than it was in 2008 (1.10).

‘Preferred’ airlines

The data gets far more interesting when customers are asked two more subjective questions, “What is your preferred airline?” and “Which airline do you think is the most passenger-friendly?”

“What is your preferred airline?”

Airline % of consumers who selected:
Southwest 18.6
Delta 9.5
Continental 7.8
United 7.5
American 7.2
JetBlue 5.4
Northwest 5.2
AirTran 5.0
Alaska 4.9
US Airways 4.2
Frontier 2.3
Hawaiian 1.3
Skywest 0.4
American Eagle 0.2
ASA 0.2
Comair 0.1
Mesa 0.1

“Which airline do you think is the most passenger-friendly?”

Airline % of consumers who selected:
Southwest 32.0
Continental 6.9
US Airways
Skywest 0.5
American Eagle 0.3
ASA 0.2
Comair 0.1
Mesa 0.1

The data in the first two charts is the cumulative result from all travelers, nearly half of whom (42.6%) took less than 5 flights a year.

When viewed from these subjective standards, the value of “the numbers” appears to have a spotty correlation with the parameters of choice.

The Regionals

The AQR report also pointed to regional airlines as being a real problem in the rankings, suggesting that as more airlines off load flights to regionals, consumer complaints will rise. Indeed, one of the researchers, Purdue’s Brent Bowen predicted that smaller aircraft usage could lead some airlines to have lower quality ratings for 2010 owing to the airline’s inability to handle seasonal spikes and bad weather.

But that is as it should be, since major airlines cancel regional airline flights faster than they cancel or delay larger jets. Still, for passengers, it is the price they pay for living or doing business in a spoke city. It is also what is pushing the move to larger regional jets as well as dual-class configurations.

Purdue’s Prof Bowen stated, "unfortunately, growth in the regional elements constitutes growth in the lowest tier of the AQR, meaning more consumers will be forced to fly airlines that have never placed higher than 12th in the ratings. Complaint levels and passenger frustration will only continue to rise as regional flights are used as a stop-gap to fix a broken system. The industry will also never be successful in terms of quality until it can be successful financially. The financial crisis in the industry is a significant factor in the decline of quality.”

In the case of the regional carriers; Mesa, Skywest, ASA, AA Eagle and Comair, there is a direct relationship between poor performance and choice. However, since these carriers invariably operate “as” a mainline operator, many travelers perhaps are unaware of the distinctions that are operative. As was seen in the Colgan crash last year, most of the traveling public assumed that the Continental livery indicated a Continental operation.

Furthermore, benefits and perks like status and FF miles are accrued not on Skywest or ASA, but on the legacy carrier for which they operate, so that most customers are unaware that they are on a different airline.

Finally, one might speculate that with their generally poor performance metrics, the legacy carriers with an increased share of their passengers traveling on these, rather than mainline flights, will besmirch the legacy for the insult. Both United and Delta, carriers that utilise lots of regional seats, saw declines in the mainline brand ranking.

The Trunk Carriers

Kudos to Hawaiian for taking the top spot for the second year in a row. Unfortunately for them, the vast majority of travelers will never use them on a regular basis due to the narrow (but well served) market niche that they occupy. One might also reasonably assume that the sampling for Hawaiian was much smaller than for, say American, which has a much broader customer base. Hawaiian also deals with very few blizzards or thunderstorms, both generally lacking from its Hawaiian and West Coast operational points.

Despite ranking fifth statistically, Southwest continues to be an overwhelming passenger favourite, with almost 1/3 of the respondents believing it to be the most passenger-friendly. As has been a steady trend, the combination of Southwest, JetBlue and AirTran encompasses just shy of 50% of the most friendly category.

In terms of preference, Southwest is again the clear winner, though the combination of Delta and Northwest is a strong second. This too is not surprising as our examination of US hubs showed that the “new” Delta not only has the most hubs but, overall, enjoys the greatest service concentration at their hub cities.

Finally, we see that the linkage between “preferred” and “most friendly” is not direct, indicating that perhaps mileage programs or service concentrations in the region determine the preferred status—as opposed to the desire for a “friendly” experience.

The Road Warriors

Anyone observing airlines knows that while the masses are nice, carriers are particularly focused on the most frequent travelers. These folks put more into the till, are more likely to be loyal and often pay higher fares.

As a result, these passengers exert far greater influence on the system than the grandma who flies twice a year, and the choices made by frequent travelers get a whole lot more attention.

The next two charts display the same information, but the respondents all fly 21+ times a year, making these opinions especially interesting to the carriers. So what do these road warriors representing just 14.2% of those responding, think?

First, like the general population, they give Southwest high marks. They also recognize the friendliness “advantage” of other new generation carriers; giving both JetBlue and AirTran high marks. But for those two carriers, that attribute does not move them up on the preferred scale.

Instead, the legacy airlines occupy the top preferred positions. As most of these frequent travelers are more likely business fliers, this is an understandable result as:

  • Corporate agreements may play a big role in their preference
  • As very frequent travelers, they are more likely to reap the status benefits granted by airlines
  • More limited, and primarily domestic route networks may not suit the travel patterns of many of these folks, pushing them to legacy networks

As other studies have shown, of the legacy airlines, Continental generally outranks its peers in terms of having a passenger-friendly reputation. These charts also display the fact that though good, Hawaiian’s limited focus keeps it way down the preferred list.

Road Warriors rankings: “What is your preferred airline?”

Airline % of consumers who selected:
Southwest 14.3
United 11.2
Delta 10.6
Continental 8.6
American 8.4
US Airways
AirTran 6.5
Alaska 4.9
JetBlue 4.3
Frontier 2.0
Hawaiian 0.8
Skywest 0.8
American Eagle 0.6
ASA 0.4
Comair 0.0
Mesa 0.0

Road Warriors rankings: “Which airline do you think is the most passenger-friendly?”

Airline % of consumers who selected:
Southwest 27.7
Continental 9.4
JetBlue 9.4
AirTran 6.5
Alaska 6.1
United 4.6
Northwest 4.1
American 3.5
US Airways 3.5
Frontier 3.3
Hawaiian 1.3
Skywest 1.3
American Eagle 0.4
ASA 0.2
Comair 0.0
Mesa 0.0

And amongst this group, more likely to know a regional from a mainline flight, the feeders get even less respect—though they did not have far to fall.

Price, hubs and more

We have long known that price is the dominant factor in choice. That was true here as well with 69.4% rating it “very important”, followed by schedule (49.7%) and customer service (48%).

In terms of creating a positive experience, 45.5% listed an on-time arrival, with customer service (42.3%) ranking second. A negative experience is most often due to customer service (34.3%) or is associated with being late, cited by 30.4%.

In keeping with our focus on hubs, 54.2% of those responding originated in hub cities -with or without much choice, depending on where one is. Addressing another current discussion, mergers, United is the top preferred network carrier among travelers with 16+ flights per year and Continental is the most passenger-friendly network carrier. In contrast, US Airways mingled with the regionals for poor passenger-friendliness. Also, passengers who preferred US Airways had the highest percentage believing that Southwest was more friendly—never a good position to be in.

One would hope that more considerations than money will enter the merger negotiations.

Outlook not rosy

Despite improved and measurable metrics, the nebulous and subjective “customer service” continues to rank as the top concern for many travelers. While things like on-time performance are certainly a part of service, Southwest leads the friendliness scores without being tops in punctuality.

Clearly the exact nature of “service” is hard to define and yet some carriers seem to have it while others, even without losing your bag, do not. Arriving on-time and with your bag is good. Arriving on-time, with your bag and a pleasant experience is better.

"Seeing a continuing upturn in quality is good news for all air travel consumers, but it does not mean we have fixed the system," said Bowen, professor and head of the Department of Aviation Technology at Purdue University, who produces the ratings with Wichita State University.

US airlines have been spending hundreds of millions of dollars to improve the customer experience. From incentive payments to employees for better performance on the monthly Department of Transportation Air Travel Consumer Report to rolling out expensive perks for their frequent flyers, they are paying attention to what their customers think.

That is why it may be somewhat disheartening to hear that, while airlines improved in the 20th annual AQR, it is largely owing to fewer passengers rather than all that investment. The annual report card clearly pointed to the capacity cuts over the last few years as the major reason for posting one of the best performance rates ever, according to the two universities.

Thus, it is entirely predictable that as traffic rises, the quality of the service will decline, just as it has in every previous up cycle. Indeed, during the last up cycle, passengers became so frustrated by the airline travel experience thus just plain opted out. Indeed, the US Travel Association said the commercial airline hassle factor drove passengers to avoid 41 million trips annually. The 2008 study said the hassle factor cost the economy USD26.5 billion which includes the USD6 billion lost to hotels and USD3 billion lost to restaurants along with the fact that federal, state and local governments lost more than USD4 billion in tax revenue.

That hassle factor was behind the incredible rise in business aviation use. Two weeks after the USTA survey, Stanford Transportation Group (STG), a leading US-based aviation consultancy, indicated that business aviation grew from 16% of all premium business traveler trips to 41%. Contributing to this is the increasing affordability of business aviation.

And therein lays the problem. The passengers in both these surveys cited government failure to upgrade the air traffic management system. Indeed, the top issue was delays. Unfortunately, the political will to do anything about modernization only comes with the increasing hassle factor. In each of the past delay crises, which gets really bad about once a decade, the economy has tanked alleviating the problem along with the political will. It is very likely that pattern will just repeat itself.

That only means that no matter how many millions the airlines spend, the quality of air service is dependent on something that is largely out of their control – the federal government.

Want More Analysis Like This?

CAPA Membership provides access to all news and analysis on the site, along with access to many areas of our comprehensive databases and toolsets.
Find Out More