How to cheat at benchmarks (and why we should downplay their importance)

ZOMG QUADRANT!!11!!

Benchmarks can be useful, whether we’re talking mobile or any other computer hardware.  There is a problem though — you have to depend on everyone being honest.  Let me explain.  Anytime you see a video or a screenshot of a benchmark score, what you’re really seeing is a measure of how the software interacts with the hardware.  There are two big problems with that: No two Android phones run the same software, and there’s always some joker who wants to cheat.  recently our pal John from Phone Arena noted some Quadrant scores of a new handset that are just too low compared to the other phones listed.  And good on John for pointing it out.

Sure, benchmarks can be fun, like see how little impact LG’s UI has on the Optimus 2X compared to Stock Froyo, and they can be useful if you’re running them yourself to gauge some changes you’re making to your own software.  But we’re over it. We’re going to base our judgment of performance based on performance, and not numbers.

It’s just too damn easy to cheat at all benchmarks, because they are just Android apps.  Look at all the other magic folks do with Android apps, and ask yourself — are benchmarks so different that they can’t be "hacked"?  The answer is no, and we’re going to show you how to hack at the easiest (but definitely not the only) one to manipulate, Quadrant. 

Check it out after the jump.


Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: