BaseMark GUI from RightWare Launches: SoCs Benchmarked

Rightware, makers of MobileMark, BrowserMark, and other mobile benchmarking tools recently launched a new 3D benchmark for mobile devices named BaseMark GUI. BaseMark joins our growing suite of mobile device benchmarks and tests OpenGL ES performance on Android and other platforms with an emphasis on 3D user interfaces. We've run BaseMark on almost all the devices we've got on hand, and will do so going forward on other devices we get our hands on.
We've been making a lot of additions to our mobile benchmarking suite lately in an effort to get the best picture possible of the performance landscape out there for tablets and smartphones alike. First it was GLBenchmark 2.0, now it's Basemark. This new test benchmarks OpenGL ES 1.0 or 2.0 performance on devices through a suite of five feature tests and one overall 3D user interface test. In its current incarnation, we've been given binaries for the Android platform, however Basemark's Kanzi 2.0 engine, which is written entirely in ANSI-C, will be ported to other platforms in the future. For now however, we're focusing on testing Android hardware with Basemark and using the OpenGL ES 2.0 render pipelines since all of the Android devices we have (and coming in the future) are of the 2.0 sort.

Basemark divides benchmarking into two categories - feature tests and the final overall user interface test. Up first are the feature tests. These are named animation, vertex streaming, texture streaming, and alpha blending as shown in the menu above. Each tests a different collection of what Rightware believes are representative graphics tasks that will be leveraged in user interfaces moving forward. The first item in that test menu screenshot above - 3D UI - is an overall graphics performance test designed to simulate a completely 3D user interface with example application interfaces. It's one of the heaviest 3D benchmarks I've seen yet.
First up among those feature tests is animation, which animates a graphics element (in this case, a robot moving through a set of actions) by stepping through a table of keyframes and interpolating the character's movement between using splines. This particular benchmark uses ES 2.0 APIs and per vertex lighting, but the purpose of this test is to be as CPU-bound as possible and specifically stress floating point performance.

Animation Test

Next are the vertex and texture streaming tests, both of which measure asset streaming performance - according to RightWare, effectively memory bandwidth. The vertex test loads geometry data into GPU memory and frees it when no longer needed. The scene itself involves a lot of geometry - ten repeating city blocks which the camera moves through with increasing speed and draw distance. The test ramps from around 3k vertices to 15k vertices per frame, and 190k to 250k triangles per frame. There's a single texture material, fog, two lights, and OpenGL ES 2.0 shaders which use per vertex lighting.

Vertex Test

The texture test is a bit more straightforward, quickly loading images from RAM into the GPU memory and discarding them.

Texture Test

These asset streaming tests effectively test memory bandwidth from disk into RAM and into to GPU memory. Although Kanzi supports ETC texture compression (and Basemark exposes this option for testing), we've initially tested with the default configuration which has ETC texture compression disabled.
Blend testing - as its name implies - tests alpha blended rendering by drawing a number of semi-transparent contact cards atop each other. These are overlaid sequentially until we reach a desired number of layers. This test actually runs twice, first with front to back ordering of these contact cards, and then with back to front ordering.
This ordering difference shouldn't be confused with the fact that feature test actually runs in both back to front and front to back rendering orders and are combined later.

Blend Test

Next is the composition test, which composites interface screens rendered off-scene into one large tiled user interface. The example here are applications rendering in the background being composited into one view. This test combines 16 off-scene objects.

Composition Test

There's also a battery test which loops the composition test indefinitely until battery dies. Going forward we will likely use Basemark's battery test to augment our three other ongoing battery test metholodogies.

3D UI Test

Last but not least is an overall 3D performance test which emulates a futuristic interface consisting of applications rendered on screens attached to a central... orb. This is definitely the most challenging of the Basemark suite, combining post processing blur for depth of field and basic blur, per pixel lighting with three point lights and one directional light, cube and normal mapping, ambient occlusion mapping, and a shader on the arms holding the displays around the cube which animates a texture map. To start we ran with 4xAA enabled (by Basemark default), though we're running Basemark GUI with AA turned off as well. Remember for a moment that all of the phones we've seen so far are tile-based renderers.