Ben Salter/flickr

Ben Salter/flickr

Applause just released its “State of the Retail Apps Economy” report card, but does it provide any real value to retailers looking to improve their apps’ end-to-end customer experience?

eMarketer predicts that [tweetable alt=”74% of digital #consumers will #shop with their #phones this #holidayseason, says @eMarketer.” hashtag=””]74% of digital consumers will shop with their phones this holiday season.[/tweetable] Depending on the retailer, these customer experiences could take place on responsive, mobile-optimized websites or on E-commerce-enabled mobile apps.

Looking at the retail app landscape, some brands are more prepared for the holidays than others. Framingham-based testing company Applause ranked this holiday season’s top retailer apps, placing eBay and Pizza Hut at the top of both the Android and iOS charts. But this report leaves something to be desired with regard to actionable insights.

Testing Methodology

As a research company that prides itself on the superiority of human testing (touching and tapping apps) over emulators, the testing methodology Applause used to produce the “State of the Retail Apps Economy” doesn’t align with the brand’s proclaimed mission.

It’s challenging for all companies to rank and compare apps in an automated way because so little information exists to inform such comparisons. The only publicly available data comes from the app store–in the form of both ratings (stars) and reviews (written in).

Layering more comprehensible and reliable data gathered from human testing and analysis on top of the app store data would have dramatically improved the quality of this report. Instead, Applause’s report scraped together some app store data based on the Nation Retail Foundation’s list of the top 100 retailers.

The Results

The final report ranks retailer apps on an index that’s based on low quality data. Usage of only app store data limits the results’ significance, as they provide no insight into what low-scoring apps can learn from those ranking higher on the list.

Specifically, the results can’t be broken down on a feature-by-feature basis, leading to generalized conclusions like, “high-ranking apps always include social media feed integration and one-click checkout.” Moreover, the report often states the obvious, emphasizing “serving customers in their mobile moment” and ensuring app stability and scalability, regardless of device, location, or network.

A retailer can best understand these rankings’ lacking informational value by comparing screenshots of top and bottom performers–as shown below. These apps are so diverse in their design, functionality, and user experience as to make the report’s conclusions on improving customer satisfaction only seem more frustratingly vague.

Top-Ranked iOS Apps

top_ranked_ios_apps

Pizza Hut, [tweetable alt=”#Report from @applause says @pizzahut, @Walgreens and @eBay have the top #retail #apps.” hashtag=””]Walgreens, eBay[/tweetable]

Bottom-Ranked iOS Apps

bottom_ranked_ios_apps

McDonald’s*, Kohl’s**, Michael’s

*There are too many McDonald’s apps in the app store to count, including multilingual and country-specific versions. In sum, the McDonald’s brand appears jumbled amongst the app store search results. The app pictured here was last updated in February 2013 and has not yet been optimized for the iPhone 5, a device released in September 2012.

**The author was unable to screenshot the Kohl’s app due to repeated crashes. This underscores Applause’s point that apps must be tested to ensure they work on all devices.

Another flaw in the report emerges in examining these screenshots: the category of “retail” is too broadly defined, inclusive of brands that range from fast food to auctions to pharmacies. Applause should consider subcategorization, as seen in the NRF’s top 100 list. Clothing retailers, for example, have more to learn from one another than from the experience of ordering pizza on a mobile app.

Other App-Measuring Indices

Despite Applause’s purported claim to the only report of its kind, other companies, such as the digital think tanks L2 and App Annie, have indexed mobile app performance and consumer preferences in similar ways.

A commonly accepted method for ranking and comparing apps does not yet exist, so it’s important for companies to keep an eye on their rank in all available indices. In many cases, a company will glean plenty of insights from monitoring their app store reviews themselves.

Advice to Retailer Apps

Applause does provide some sound advice regarding customer experience: the apps that will “win” this holiday season are the ones that will provide a seamless interaction for every customer, every time.

As the report somewhat redundantly states, stability and scalability are key. For retailers still updating their apps throughout the holidays, or developing their own with platforms like Infinite Monkeys, keep in mind the importance of app monitoring and testing. You always want to catch user experience issues before they are reported to the app store and count as a “ding” against your brand in reports like these.