Android Graphic rendering true facts

How about some Android graphics true facts?

I get tired of seeing so much misinformation posted and repeated all over the place about how graphics rendering works on Android. Here is some truth:

• Android has always used some hardware accelerated drawing. Since before 1.0 all window compositing to the display has been done with hardware.

• This means that many of the animations you see have always been hardware accelerated: menus being shown, sliding the notification shade, transitions between activities, pop-ups and dialogs showing and hiding, etc.

• Android did historically use software to render the contents of each window. For example in a UI like http://www.simplemobilereview.com/wp-content/uploads/2010/12/2-home-menu.png there are four windows: the status bar, the wallpaper, the launcher on top of the wallpaper, and the menu. If one of the windows updates its contents, such as highlighting a menu item, then (prior to 3.0) software is used to draw the new contents of that window; however none of the other windows are redrawn at all, and the re-composition of the windows is done in hardware. Likewise, any movement of the windows such as the menu going up and down is all hardware rendering.

• Looking at drawing inside of a window, you don’t necessarily need to do this in hardware to achieve full 60fps rendering. This depends very much on the number of pixels in your display and the speed of your CPU. For example, Nexus S has no trouble doing 60fps rendering of all the normal stuff you see in the Android UI like scrolling lists on its 800x480 screen. The original Droid however struggled with a similar screen resolution.

• "Full" hardware accelerated drawing within a window was added in Android 3.0. The implementation in Android 4.0 is not any more full than in 3.0. Starting with 3.0, if you set the flag in your app saying that hardware accelerated drawing is allowed, then all drawing to the application’s windows will be done with the GPU. The main change in this regard in Android 4.0 is that now apps that are explicitly targeting 4.0 or higher will have acceleration enabled by default rather than having to put android:handwareAccelerated="true" in their manifest. (And the reason this isn’t just turned on for all existing applications is that some types of drawing operations can’t be supported well in hardware and it also impacts the behavior when an application asks to have a part of its UI updated. Forcing hardware accelerated drawing upon existing apps will break a significant number of them, from subtly to significantly.)

• Hardware accelerated drawing is not all full of win. For example on the PVR drivers of devices like the Nexus S and Galaxy Nexus, simply starting to use OpenGL in a process eats about 8MB of RAM. Given that our process overhead is about 2MB, this is pretty huge. That RAM takes away from other things, such as the number of background processes that can be kept running, potentially slowing down things like app switching.

• Because of the overhead of OpenGL, one may very well not want to use it for drawing. For example some of the work we are doing to make Android 4.0 run well on the Nexus S has involved turning off hardware accelerated drawing in parts of the UI so we don’t lose 8MB of RAM in the system process, another 8MB in the phone process, another 8MB in the system UI process, etc. Trust me, you won’t notice -- there is just no benefit on that device in using OpenGL to draw something like the status bar, even with fancy animations going on in there.

• Hardware accelerated drawing is not a magical silver bullet to butter-smooth UI. There are many different efforts that have been going on towards this, such as improved scheduling of foreground vs. background threads in 1.6, rewriting the input system in 2.3, strict mode, concurrent garbage collection, loaders, etc. If you want to achieve 60fps, you have 20 milliseconds to handle each frame. This is not a lot of time. Just touching the flash storage system in the thread that is running the UI can in some cases introduce a delay that puts you out of that timing window, especially if you are writing to storage.

• A recent example of the kinds of interesting things that impact UI smoothness: we noticed that ICS on Nexus S was actually less smooth when scrolling through lists than it was on Gingerbread. It turned out that the reason for this was due to subtle changes in timing, so that sometimes in ICS as the app was retrieving touch events and drawing the screen, it would go to get the next event slightly before it was ready, causing it to visibly miss a frame while tracking the finger even though it was drawing the screen at a solid 60fps.

• When people have historically compared web browser scrolling between Android and iOS, most of the differences they are seeing are not due to hardware accelerated drawing. Originally Android went a different route for its web page rendering and made different compromises: the web page is turned in to a display list, which is continually rendered to the screen, instead of using tiles. This has the benefit that scrolling and zooming never have artifacts of tiles that haven’t yet been drawn. Its downside is that as the graphics on the web page get more complicated to draw the frame rate goes down. As of Android 3.0, the browser now uses tiles, so it can maintain a consistent frame rate as you scroll or zoom, with the negative of having artifacts when newly needed tiles can’t be rendered quickly enough. The tiles themselves are rendered in software, which I believe is the case for iOS as well. (And this tile-based approach could be used prior to 3.0 without hardware accelerated drawing; as mentioned previously, the Nexus S CPU can easily draw the tiles to the window at 60fps.)

• Hardware accleration does not magically make drawing performance problems disappear. There is still a limit to how much the GPU can do. A recent interesting example of this is tablets built with Tegra 2 -- that GPU can touch every pixel of a 1280x800 screen about 2.5 times at 60fps. Now consider the Android 3.0 tablet home screen where you are switching to the all apps list: you need to draw the background (1x all pixels), then the layer of shortcuts and widgets (let’s be nice and say this is .5x all pixels), then the black background of all apps (1x all pixels), and the icons and labels of all apps (.5x all pixels). We’ve already blown our per-pixel budget, and we haven’t even composited the separate windows to the final display yet. To get 60fps animation, Android 3.0 and later use a number of tricks. A big one is that it tries to put all windows into overlays instead of having to copy them to the framebuffer with the GPU. In the case here even with that we are still over-budget, but we have another trick: because the wallpaper on Android is in a separate window, we can make this window larger than the screen to hold the entire bitmap. Now, as you scroll, the movement of the background doesn’t require any drawing, just moving its window... and because this window is in an overlay, it doesn’t even need to be composited to the screen with the GPU.

• As device screen resolution goes up, achieving a 60fps UI is closely related to GPU speed and especially the GPU’s memory bus bandwidth. In fact, if you want to get an idea of the performance of a piece of hardware, always pay close attention to the memory bus bandwidth. There are plenty of times where the CPU (especially with those wonderful NEON instructions) can go a lot faster than the memory bus.


Steve kondik