In the mobile marketplace, high scores on recognized application benchmarks are a must for a successful product launch, but how well do these benchmarks reflect actual user experience?
We’ll compare how benchmark hotspots measure up against popular application hotspots on several multicore Android devices and assess the relative importance of java, native, browser, and graphics components. How should benchmarks best be used to guide platform optimization for optimal user experience? Not all benchmarks are multicore mature; is it reasonable to exploit a subtle multicore bug to improve a benchmark score?
Ultimately, benchmark scoring requires a sensitive balance between business, ethical, and user experience goals.