Posts Tagged ‘Masabists’

Masabists: RSS Feeds for the Community

// March 4th, 2008 // Comments Off // Mobile

As an RSS junkie, I get almost all of my mobile information through news feeds. Unfortunately, a few key sites just don’t have them – most frustrating.

Well, after a little research I discovered the excellent Feed43.com, which I have been testing for the last few weeks. Their service allows you to rip a feed from any site, but offers far more control than any of the competitors I could find.

As a service to the mobile community we have decided to make these feeds available to anyone who needs them – on a non-commercial, unsupported (but hopefully as reliable as possible) basis, of course! The sites we have ripped feeds from are selected for their own quality, we aren’t claiming any rights or ownership over the content and we urge you to support them in whatever way they want.

One last note: there are individual feeds offered direct from Feed43, and aggregated feeds aliassed through Feedburner. If anythign serious happened with Feed43′s service we might consider changing the URLs of these individual feeds, but we will update the aggregated feeds accordingly and the Feedburner URLs will remain the same.

Without further ado – the feeds:

Of course, while you’re in an RSS mood remember to bookmark the Masabi feeds as well!

I hope you find all these feeds as useful as I do!

Please note: occasionally, the Feed43 feeds seem to throw up “Error: Source Page not available” news items; just ignore these, over the last few weeks of testing I’ve found that the service does work nicely most of the time despite occasional hiccups!

Masabists: The Truth About Mobile Fragmentation

// January 14th, 2008 // Comments Off // Mobile

Note: this was first published on the Masabists blog, and was featured on Carnival of the Mobilists #107 and #108 (not sure why it reached both!).

My most recent task has been architecting the Playtech mobile product, which currently encompasses 10 games running on over 600 devices in 8 languages, each heavily customised for multiple licensees. Furthermore, the system was designed to continue scaling across more than a hundred licensees, with dozens of games and languages, supporting all mass market devices, all managed by a small team of inter-disciplinary experts. This has given me some interesting insights into the problems of fragmentation which I would like to share.

The dominant UI model on the desktop has been the Windows, Icons, Menus and Pointers system that became popular when Apple copied it from Xerox, and has stayed in the mainstream since Microsoft copied it from Apple.

No such standardisation is present in the mobile handset world. This will remain the case for the foreseeable future, as the market segments – offering “Swiss Army Knife” phones for power users, as well as specialised phones for people who value music, games, photography or fashion most.

Some will have keyboards, some trackballs, some touchscreens; some will have high resolution displays, some low; some will be landscape oriented, some portrait; some will have powerful processors and lots of memory, some won’t; and all of this will change radically over time, with some users upgrading every six months and some every four years.

Factor in the tight commercial deadlines that phone releases are subjected to, which never leave enough time for firmware QA, and you see why any mobile development platform that is large enough to be mainstream will be fragmented for a long time to come.

Anyone who has ever attempted to develop software for mobile phones knows that there are many hidden pitfalls along the way. However, mobile development need not be fraught with pain. A little experience immediately allows you to identify the key problems before you write a line of code, and design around them; a few rules of thumb can work wonders on development schedules and keep the QA team happy.

Pick Your Platform

I’ll try not to repeat my earlier post here, and will just briefly describe the three main types of mobile application platform – SMS, browser-based and installed application. Each has its own strengths and weaknesses, and they can often be used together to build a stronger product.

SMS is excellent for short, simple, immediate interactions across any handset – but break down once complex instructions are required.

Browser-based applications are growing hugely on the fixed web now that client-side scripting (such as AJAX) allows slick interaction, benefiting from automatic updates for every user – because everything is fetched from the server on-demand. AJAX on mobile is more theory than reality right now however, able to reach a tiny fraction of the market with incompatible implementations. Browser rendering is also hit or miss, and data costs can rapidly rack up as XHTML markup is heavy and flat rate data rare. Data cost is still an issue, with prepay customers regularly overcharged; flat rate, when available, is often full of loopholes and obscure clauses – billing per Kb is still very much the norm outside of the US.

Browser fragmentation will get worse long before it gets better – for example, radically different hardware like the iPhone introduces special case UI and design rules, and then 3rd party browsers like Opera or Mozilla further multiply these special cases whenever they launch for that hardware.

The other option is an installed client-side application, offering immediate and offline access, far greater presentation opportunities, and the potential for reduced data costs – but it is harder to update over time, and is best suited to services which are intended to be used regularly.

The Traditional View of Fragmentation

The most common thing people will say when discussing mobile applications, particularly those which are installed on the client-side, and Java apps in particular, is that fragmentation is a huge problem. This is generally repeated as a fact even by those who have never coded Java, which is why it is so often misunderstood. Figures like the 25,000 builds Glu required for Transformers are repeated with disbelief (which I share); I’d love to know how many of those included the tiny changes that operators demand to feature content on their portals, but I’d wager it’s a huge factor because I know Glu aren’t incompetent. As a comparison, we normally run to between 30 and 60 builds per language for an app or game.

Some history may explain how this feeling became “common wisdom” in the industry. In the bad old early days of MIDP 1 (the first version of mobile Java), applications were quite restricted in what they could do. MIDP 1 did not define a way to play sounds, vibrate the handset, or provide a mechanism to allow the game to take over the entire screen, for example.

Most major handset manufacturers saw this as an opportunity to differentiate their products, and dived in with proprietary APIs to do lots of extra things which arguably the MIDP 1 spec should have included in the first place. These could all be abstracted away by developers behind a common internal API and a single build, but it would have required a little effort to set up and was hardly ideal. This led to much of the initial bad press for mobile Java, which is still heard to this day. Buggy handset firmware is of course the other huge factor, but it is hardly restricted to the development environments – whole handsets regularly suffer.

Sun (the makers of Java) eventually got their act together and started to introduce extended functionality through the JSR process: first MMAPI for sound, then WMA for text messaging, and then MIDP 2. Rapidly, the old manufacturer APIs dropped away; some can still be found, but they are really only objects of curiosity these days. Right now, MIDP 2 is available on over three quarters of actively used Java-enabled handsets, with most popular devices also featuring lots of the common extensions as well.

The next most popular platform, Flash Lite, is now available on commonly used handsets in versions 1.0, 1.1, 2.0, 2.1 and 3.0. All of these offer different capabilities and implementations run in one of two different ways: either embedded on a web page (like a traditional Flash movie) or running as a standalone app (like JavaME content), each of which brings critical differences to the experience. Japanese handsets (where Flash Lite is arguably most popular) tend to embed Flash in the browser, preventing the Up and Down keys from being used in the Flash content. The main reason Flash Lite has fewer variations than JavaME is that it offers less power and access to the device’s features – and it still has its fair share of the standard cross-device issues and bugs.

After Flash come native Symbian OS apps – offering plenty of APIs (between the three major and many minor versions of Nokia’s Series 60 and the keypad and multiple touchscreen variants of UIQ). Recent versions of Symbian have repeatedly broken binary compatibility and the range of APIs always expand alongside device capabilities. So it goes on.

Mobile development platform graph

It is inevitable that any new feature absorbed into a phone – from cameras and music playing to GPS location tracking, OpenGL 3D and operator micropayment integration, will require a dedicated API before developers can access them. Features are being absorbed into phones at a very rapid pace, so any living development platform, be it browser or fully fledged open OS, will have API growth over time. There is simply no avoiding it.

Loosely drafted specifications with optional features and optional supported file formats still require management and should be avoided, but this is not rocket science. If you know that some devices support WAV sound but others only handle AMR, it’s not a particularly difficult problem to set up builds for the former to get WAVs and builds for the latter to get AMRs. We now have to generate platform-specific builds for a single application – we have certainly lost the “Write Once, Run Anywhere” bonus we were promised with Java. However this is simply unavoidable in the mobile space whatever platform you use, once you move beyond text-only messaging.

For what it is worth, my personal feeling is that the core Java functionality is becoming more reliable over time – we haven’t had anything as bad as a 7650 or a 6600 for some time, and even some of the more notorious platforms like JBed are getting more stable. The more obscure APIs have teething problems and no platform is bug free, but most everyday features can be used easily enough once you know the basic tricks.

Mine’s Bigger than Yours

I’ve just listed one reason we may wish to split a single application into multiple binary files: incompatible file formats between devices. There are many more, and most apply to any development platform you care to mention.

The single factor which leads to the most extra work on any mobile app is the size of the device’s screen. The smallest screen the Playtech mobile product supports is 96×65 pixels for Nokia’s still popular low-end devices; the current largest is 352×416, used on some of Nokia’s highest end handsets; I have no doubt we will expand to support VGA and larger screens soon enough. In between, there are currently eight other screen size and orientation groups, all requiring individually redesigned graphics, to cover a few dozen actual real screen resolutions. For example, we consider a 176×196 pixel usable screen area to be equivalent to 176×220, and design the graphics and code to be happy with any height between 196 and 220px.

Note that often, the phone will let applications take over most of the screen, but keeps a header and/or footer for showing battery and signal information, the time etc.

Mobile apps and games almost exclusively use bitmap graphics – which have a fixed size and resolution – and not scalable vectors, like a traditional Flash movie, that would be capable of adapting somewhat to different screens sizes from only one binary file. Even Adobe admits that Flash Lite will need different binaries for different devices, as whilst Flash Lite theoretically offers the same vector scaling as its big brother, in reality this is too slow on current devices for fast complex graphics. Experts recommend the use of bitmaps for any non-trivial application, just like Java.

Managing this screen size complexity is a challenge without huge graphics production timelines (and suicidal designers), but one that can be met with scripting and templates. In Playtech’s case games can feature amazing levels of variation where the graphics require less than a day to customise for a new licensee, ready to drop into an automated build.

   

Fragmentation for All!

Assuming that intelligent code, good tools/scripting and carefully planned graphics can cope with screen size variations and differences in processor speed, what else do we have to worry about? Principally, the very first point mentioned in this post – the UI model of the handset, and the input mechanisms provided to make use of it.

The iPhone was rightly lauded for its touch user interface, a radical departure from even the conventional stylus-powered PDA platforms like UIQ and Windows Mobile, and thumb-powered interfaces as featured on the LG Prada. Windows Mobile itself is now changing to a new model based on gestures, and Nokia’s Series 60 is getting touch screens as well. Great news for consumer choice, but a fragmentation headache for developers – each of these systems requires its own dedicated interface solution to best meet the user’s expectations.

Even on more conventional phones, there are plenty of other differences – to enter some text into a field, most provide simple numeric keypads, but some handsets provide full keyboards and some provide their own proprietary solutions, such as RIM’s Suretype system. Even with a simple numeric keypad, some devices have a dedicated Clear/Delete key and some require a soft key for this function. In theory, you could just create a Java text field and let it worry about this detail, but that would tie you down to very ugly screens with extremely limited control. Most quality applications will implement their own UI components with full branding, but this means extra effort to support all types of character entry.

Even an app with no need for text entry should optimise the rest of the interface for whatever keys are available – allowing users to jump through lists by typing, for example, or handling motion control in games on devices which have no dedicated Up/Down/Left/Right keys. The optimal UI for a device with an Up/Down scrollwheel may be very poor on a device with a trackball, etc.

For a trivial app, these differences do not matter – and in a webapp, you simply can’t do these optimisations for your user experience. For a professional installed app that will see frequent and prolonged use, however, these optimisations are 100% critical to the project’s success. The interface must react exactly how the user expects it to, looking and feeling familiar whilst also presenting an attractive, professional and branded appearance.

OS Wars

Many years after Symbian was formed to be the phone OS of the future by some of the key mobile manufacturers of the day, the marketplace is still very fragmented. Motorola currently supports MIB, AJAR, MAGX (a Linux variant), Windows Mobile and UIQ (a Symbian variant); Samsung relies on a number of internal platforms alongside Windows and S60 (the other open Symbian variant); Nokia, the truly dominant force in mobile with between 30% and 40% of the global market at any given time, has managed to rationalise its OS portfolio down to the low-mid range Series 40 and the S60 platform for smartphones. Some operators are attempting to limit the number of platforms they support, but this is starting to look like trying to hold a finger in a leaking dyke.

If Nokia, by some metrics the most successful consumer electronics company the world has ever seen, finds it very hard to raise market share beyond 40%, then it would seem unlikely that anyone else can achieve more. Slow handset upgrade cycles mean that even if they did tomorrow, today’s fragmented handsets would still be in regular use in 2010. Current marketplace trends see increased diversification, both in form factor and in the types of software running on it – and with more software development platforms to work with. Fragmentation may be annoying, but it is here to stay.

Conclusions

Mobile development will always involve hitting multiple moving targets, and the only applications which succeed will be those that embrace this complexity and manage it sensibly. It is more important to ask “how many people can a platform reach?” than “how few targeted binary files will I have to generate?”

The only serious platforms that can give a satisfying answer to this question today are clearly WAP2 or Java, depending on the nature of the application. One day, it will hopefully become possible to write scriptable mobile web applications that can be updated day to day from the server end, but that simply is not possible for the mass market today, and won’t be for some years.

As CTO of a company which is in the business of knowing the ins and outs of every handset and every mobile platform, it is obviously to my advantage to state that only developers who can manage this complexity will be able to survive and thrive in the mobile space. Hopefully, the reasons why this is true, and the reasons why we as a company have taken the sometimes painful direction that we have, are clear. This is a very exciting space to be in, but when you try to work on the cutting edge you must take precautions not to get cut.

Please leave comments on the original post.

Masabists: Cell IDs and Location-Based Services

// December 1st, 2007 // Comments Off // Dev, Mobile

Note: this was originally posted on the Masabists blog.

It was very interesting to read about the latest update to Google Maps, one of the nicest J2ME apps around at the moment, which can now find your location without GPS. My instant reaction was – “Finally! But will the operators let them continue?”

Current operator location services work by triangulating signal strength from multiple base stations, which can often give good accuracy in urban areas densely packed with cells. They carry with them a cost – low in absolute terms but sadly quite high for a lot of possible use cases – and all sort of privacy controls, which whilst clearly necessary have been a bit of a barrier to widespread adoption of Location-Based Services.

Back in I think it was 2002, Masabi had a working system to track handset location by cell IDs. Ben, being an engineer at heart, had strapped a modem unit to a Palm PDA and written an application to read out the current cell ID and plot it onto GIF maps downloaded live from StreetMap.co.uk. I distinctly remember being very impressed walking down Victoria Street towards Parliament Square in Westminster and seeing it track us across the map on this very GIF with surprising accuracy.

Consensus seems to be that Google are using a very similar system, with GPS users providing location data to map out operator’s cell IDs (something I believe explicitly mentioned). This suggests that Google haven’t purchased the location data from the operators. Why would that matter?

So how did we build up our cell location database? And if it worked, why didn’t we commercialise it? The two answers are connected – we were ramping up for a launch within certain industries which could have benefitted from a single network/limited device range service. Unfortunately – or perhaps fortunately, with hindsight – just before a major demo, the operator we were using decided to remove the cell broadcast info that had been supplying the base station OS grid reference locations (note: the cell IDs themselves did not appear to change, as I had erroneously stated earlier).

We considered some sort of effort to map cell IDs into a database, perhaps open source, but without widespread GPS ownership this was a huge task and there was no guarantee that the operators wouldn’t choose to change the IDs at any time in the future and we were not interested in trying to make commercial promises where we had no control over key components. So we put it to rest.

Some JavaME devices can access the current cell ID, as can signed Symbian apps and Windows Mobile apps; Google’s compatibility list suggests they are targeting only these devices, suggesting they are attempting something similar. I wish them luck!

Please comment on the  original post.

Masabists: Thick vs Thin Clients In Mobile Today

// August 10th, 2007 // Comments Off // Mobile

Note: this was originally posted on the Masabists blog.

There are three ways to handle data services with the owner of a mobile phone – via SMS, with a wap site (a ‘thin’ client) or with actual code running on the device (a ‘thick’ client). Each has its place.

SMS is an excellent way to transfer small amounts of data, and it can be monetised if that’s part of the plan. Users can submit simple multiple-choice selections or whole messages using their standard phone functionality in a familiar way – all they need to do this is a short number. I don’t need to tell you what this is good for as the marketing and media communities have pushed the medium to its limits, but it is worth pointing out what it is bad for:

  1. Security. Operator data centres can be hacked, and you should assume that the contents of any message could be intercepted. Phone theft is an even greater risk, with the inbox/outbox potentially a treasure trove for thieves if some of the more foolhardy plans for SMS services ever get to see the light of day.
  2. Complex data entry. An SMS is great for a one letter answer to a quiz. It’s good for asking a free form question from a service like AQA. Entering carefully structured data is simply too much for most users though, so any service which requires the user to remember arcane data markup just isn’t going to cut it, especially if there is a large amount of data to enter.
  3. Trust. That’s taken a hammering recently and it may take a while for confidence to come back.

SMS apps are a very well ploughed furrow so I’ll concentrate the rest of this post on the other two options – the thick and the thin client.

A typical thin client could be a news or weather wap site, or a wap-based email system; an advanced system might be a wap version of an eCommerce site. All interaction requires an active network connection and takes place through a browser using the limited form fields available in Wap/XHTML-MP (a subset of normal web forms).

A thick client, on the other hand, is a downloadable application which must be installed onto the handset and stored locally, but can then be run at any point – a network connection is only needed if it needs to interact with a server, and the interactivity options are limited to the imagination of the programmer and the physical constraints of the device.

We should also mention briefly at this point the halfway house, Mobile AJAX. This aims to offer the advantages of thin client browser-based content (no installations, instant access to new features, etc) but with the greater interactivity of a thick client. It is a worthy goal hampered by a long list of constraints, principally very limited handset support and an architecture that is suited to cheap high-bandwidth low-latency high-uptime connections, everything a cellular connection is not. There are plenty of in-depth critiques of the platform as it stands today if you want to read further.

So if, in today’s mobile environment, both pure thick and pure thin clients have their strengths then what might those strengths be?

Thin clients are good for:

  • Displaying regularly changing information eg. news.
  • Casual one-off browsing, where the user is unlikely to ever need to come back and benefit from having any sort of state remembered, and not being able to access any content is not a serious problem (due to lack of signal, etc).
  • Very changeable data with simple display requirements that change unpredictably over time, with minimal data entry requirements.
  • Situations where reliable network connections are guaranteed – either fast connections, or very small nuggets of content that can be found very quickly.

Thick clients are good for:

  • Repeated user experiences – local secure caching can reduce data entry, always a significant pain point on mobile.
  • Intelligent data delivery can reduce costly data overheads whilst providing faster responses for the user for known types of data set.
  • High interactivity and improved user experiences with many potential data visualisation models and on-client validation (minimising round trip requirements to the server, which cost money and time).
  • Improved branding around content, which only ever has to be downloaded once rather than at the start of ever session – handsets rarely dedicate persistent storage to browser caches so ever visit to a branded site reloads many Kbs of images, CSS etc.
  • Real security, guaranteed – there are known issues with Wap security which can even affect Wap2 browsers running through legacy APNs.
  • Offline access – content can be stored locally to browse even when there is no signal (eg. Tube maps), data entry can be performed when required and then submitted to a server asynchronously etc.
  • Data roaming can be incredibly expensive so for content like travel guides designed to be read abroad, or interactive services for which Java can choose to use SMS instead of GPRS, there are big cost savings to be had.
  • Precise optimal rendering of certain types of content, for example barcodes (used for ticketing etc) which would be useless if the handset decided to rescale them badly as can happen when displaying MMS messages or images in browsers.

We can use these criteria to draw up a number of ideal candidates for each type of application – thin clients are great for news sites, cinema listings and other casual browsing whilst thick clients are better for things like online banking, repeated ticket purchases, eBooks and the like.

There is a very large grey area between these groups of ideal candidates, where exact circumstances could favour one approach or the other. E-mail, for example, is an ideal thick client app until networks become fast, cheap and reliable enough for a web-based approach –not having to install anything could outweigh the data overhead of resending the entire UI in every page, if offline access is not important, etc.

A dedicated analysis would be helpful for most projects, and the outcome of this analysis will depend on factors like:

  • Cost sensitivity, for example with data volumes and the overhead of resending branding and UI with every page.
  • The importance of a good UI and the complexity of the content.
  • Whether the project must be deployed now or can wait some number of years for the technology to catch up (3G penetration hit about 10% in most of the developed world at the end of 2006, so this can be a slow wait!).

I think most of the thin client cases are pretty obviously good as thin clients, but we can look at a little example which might help demonstrate some of the advantages of thick clients – here using my Barclays bank statement, simply because I happen to have a Barclays bank account (there is nothing special about Barclays in this context).

When I access my Barclays statement through the ‘normal’ web in Firefox, the HTML page is 98Kb (plus an extra 20Kb or so of graphics and CSS). This is the same page, incidentally, which the user of an iPhone or the WebKit browser on a Nokia Series 60 handset would enjoy downloading. The total page size is 118K * 1024 * 8 = 966,656 bits which at an absolute peak rate of 384Kb/s would take 2.5 seconds to download on the iPhone’s EDGE connection – real world performance would probably be under half that for the download, once we remove the fixed overheads of initiating a connection to the network etc. Nokia users lucky enough to be on a 3G network could get it quicker.

I had started this article hoping to use Barclays new mobile service – “specially redesigned to be viewed on mobile phones” – as my thin client example. Sadly right now Barclays.mobi appears to redirect users to the standard web site, so I’ve had to manually recode the HTML page as efficient XHTML-MP to estimate the size of the same statement if generated as a proper mobile-optimised thin client – it comes out at between 20-30Kb depending on exactly how streamlined you want to make the browsing experience, because the Wap2 XHTML tags are transmitted plaintext and add a lot of padding around content (Wap1 WML is converted to smaller binary which would help).

Barclays also recommend you use Opera Mini, a Java-based browser which recodes and compresses pages on a server into a proprietary Opera format before passing them to the client. This approach crunches the same page down to 7Kb, though on the UI front you see that even with the best mobile refactoring of the page you still have to scroll through 3 screens of nav links etc to actually reach the top line of the statement. It is worth noting that Opera themselves do not recommend using Mini for passing any kind of secure financial information around, something we touched on in a previous post.

If we now look at Masabi’s thick client banking app concept, we see two things – firstly, that same statement can be encoded in around 0.5Kb, and secondly the initial statement lines are completely visible with the display optimised for the user to instantly reorder statements whilst making best use of the available pixels. The initial download for the client would be 50-100Kb, depending on the level of branding the client used and the type of handset – about the same as the HTML for one normal web statement in fact. Here is a quick screenshot comparison:

Banking comparison of thin and thick client screens on mobile

Why do a few kilobytes matter? Well besides the speed of response – which can be enough to kill an app for some users – there is also a rather large cost factor associated with mobile. Despite all of the recent talk about flat rate data, flat rate really means quite a large fee for quite a small cap under the operator’s “fair usage” policies (surely a target for an enquiry into misleading nomenclature). Orange’s recent attempt offers a 30Mb cap for £8 per month – if we ignore the image/CSS overhead 30Mb is 300 full-web bank statement views, though in reality you’d need to go through a multi-step login and view a few other pages every session, so you’d maybe get at most half that. No-one is going to need to see their statements that often, but the general point here is that un-optimized data usage adds up very quickly if the user accesses a few services, and that can easily lead to bill shock. Orange are competitive but they don’t have the most generous data bundle in the UK right now – but then most customers don’t have a data bundle at all; Orange customers without a bundle pay £1.18 per Mb when in UK (£7.64/Mb roaming abroad), whilst O2 PAYG customers pay £3/Mb (in the smallprint). It will be a long time before proper data bundles are offered by the operators, and longer still before the average consumer sees any point in buying them if they are put off mobile data by early billing shocks.

A payload of half a kilobyte, in comparison, barely registers and can be downloaded very quickly; the Masabi client also has the benefit of real end-to-end security on all handsets regardless of certificates.

Comparison of ways to deliver bank account statement to mobile

A thick client can be implemented badly, and could easily throw away all the potential usability and bandwidth gains we have discussed here. Web content can be manually fine tuned to be much more usable than the Opera Mini example above, and some types of data displayed on the web will translate better into mobile browsers than others. However when implemented with care and attention, for the right types of application a thick client makes a huge difference to the end user experience and cost and that has a huge impact on the uptake of the product.

Please comment on the original post.

Masabists: A Mobile Software Primer

// July 4th, 2007 // Comments Off // Mobile

This post was originally posted on the Masabists blog.

I originally started this post to talk about mobile statistics, and particularly the difficulties I had coming up with a nice graph to show the relative popularities of the different mobile software platforms – Java, Flash Lite etc.

Mobile Software Platform comparison graph

The more I wrote, the more I realised that what I was writing made no sense without background information – and before I knew it, most of the post was background. So I changed the title and now have what I feel to be quite a fitting first serious blog post – a primer document explaining a little about the world of mobile software.

The Mobile World

The mobile world is not homogenous. The first big split to be aware of is the separation between the GSM and CDMA network technology families, with CDMA popular in the US and GSM used in most of the rest of the world (~73% in 2005). However, GSM is used in the US and CDMA is used in some other countries, so it’s hard to be precise!

Many CDMA phones use the BREW platform for downloadable content, whilst most GSM handsets capable of accepting 3rd party software support Java. BREW content must go through operator approval and is not suitable for guerrilla marketing or free content; Java content can generally be written and downloaded by anyone, though in some countries (eg. US) some networks choose to limit its power or availability.

Two countries stand out as being completely different: Japan and South Korea, the two most advanced mobile markets in the world. Japan is split between three operators pursuing different strategies, by far the largest being DoCoMo with its own flavour of Java, DoJa, that is open for any developer; KDDI use BREW and Softbank use standard MIDP Java but all content must be approved by them before installation. South Korea has a government-backed software platform, Wipi, available only in Korean and subject to tight operator controls, effectively prohibiting non-Korean companies from accessing that market.

The US also stands out, because it is a complex mix – its largest networks include a CDMA network where BREW is used, Verizon; a CDMA network using Java, Sprint; a handful of GSM networks (AT&T, T-Mobile); and many smaller networks using a range of network technologies. Software distribution is therefore also complex.

One final thing to note is that different platforms have different strengths around the world. BREW is mainly a US-focussed technology, though it is available on at least one network in “30+” countries round the world – because of the closed distribution we won’t deal with it much here. Palm is virtually unheard of outside the US; Nokia’s smartphone platform, Series 60 (based on the Symbian operating system) is much stronger in Europe, and Japan is dominated by DoCoMo’s decision to base all of its handsets on either Symbian or Linux buried underneath their own abstraction layer. Handset use is similarly varied – SMS has traditionally been unpopular in the US, the Japanese use e-mails instead of text messages, etc. Never make assumptions!

Software Platforms

The platforms mentioned so far, Java and BREW, are by far the biggest. Here’s a full overview of the competing technologies (note for non-technical readers: it does get a little technical in places and can certainly be acronym-heavy, but hopefully it should still hold together if you skim past the bits where your eyes glaze!):

  • Java – in almost all the world this means MIDP, a ‘profile’ of Java Micro-Edition designed for phones. When people say “Java content”, they almost certainly mean MIDP JavaME.
    • Java content is downloaded through a browser, and then lives on the phone; it can be run at any time, even when the network is not connected;
    • There are numerous extensions to Java to do special things like take photos with the camera, display 3D graphics and make payments using your phone bill – availability of these extensions varies;
    • Distribution can be through a browser and often also over Bluetooth/Infrared or through a PC sync cable.
  • Flash lite – a cut-down variant of Flash available on some high-end mobile phones, this comes in two semi-compatible versions (1.x and 2.x) and can be run in three different ways:
    • Some handsets only support standalone content (like Java);
    • Some handsets only support Flash embedded in a browser page (like a PC), and often the browser prevents the Flash from using the direction keys which instead scroll the page (common in Japan);
    • Some handsets allow Flash to be used on menus and screensavers, whilst some only use it on menus (like the Samsung D600).
  • Symbian – the biggest smartphone platform, many Symbian OS handsets can accept content written in C++ that can access a wide range of platform features.
    • New versions of the operating system have not been backwards compatible, so software must be recompiled and must now be sent to Symbian for ‘signing‘ before distribution is possible;
    • There are two main variations of Symbian in the West – Series 60 promoted by Nokia, and UIQ now owned by Sony-Ericsson; a third version is sold in Japan and does not accept any Symbian content at all.
  • Windows Mobile – Microsoft’s mobile operating system has grown from very humble beginnings into a respectable platform, and much like Symbian can be extended with C++ code or a cut-down version of .Net. Backwards compatibility is better than Symbian.
  • Linux – there is no single “mobile Linux” (yet); currently, there are a wide number of variants, some of which are open to 3rd party software and some of which are locked down and cannot accept any.
  • Mobile AJAX – the iPhone has popularised the idea of AJAX on mobile phones, but despite promise for the future this is in its infancy
    • One day it should be viable on mobile networks, but this will require flat rate data tariffs and much faster connections;
    • There are a large number of browsers, all with subtly different implementations, and no good frameworks yet to abstract them and make development easy;
    • For a full critique see here (among others).
  • Widgets – a number of companies are pushing widget platforms for mobile, notably Nokia with the Java-based Widsets that run on a lot of high-end phones.

One key thing to remember here is that many of the criticisms aimed at one platform or another are actually just common to mobile phones in general – in particular fragmentation and device bugs.

The mobile handset is one of the fastest moving, most complex consumer electronics markets in the world – a bewildering array of devices are released every month, all with differing specs, and all with release dates specified by the marketing department who have booked advertising well in advance. This is a QA department’s nightmare, so handsets ship with bugs – and firmware upgrades are risky and cumbersome, so they often go unpatched. Every platform will have bugs; if it hasn’t got many yet, it is too young and too small.

Even if there were no bugs, fragmentation is guaranteed when you have such a bewildering variety of screen sizes and methods of communicating with the device. Some have touchscreens, some numeric keypads, others full QWERTY keyboards or custom thumbpads. The Nokia N95 has a 5Mp camera, the E61 has none; the Motorola Z8 can expand to 32Gb of storage, the original RAZR has 5Mb. Sony-Ericsson use dedicated Back and Clear keys, Nokia phones have neither. Even the layout and usage of basic keys can be different. Here is a diagram comparing all of the resolutions of screen Nokia currently has on the market – just try designing graphics for a single app, or one wap site, to run on all of them:

Current Nokia screen sizes

These problems are manageable if you understand them in advance – though more so on a mature platform like Java (sticking to core APIs) then on a bleeding-edge beta platform like Mobile AJAX.

Lots of Platforms – But Who Can They Reach?

Mobile handset statistics are notoriously difficult to obtain. Starting from a blank slate, you have three basic choices:

  1. Pay a lot of money for comprehensive breakdowns of some key markets, from research companies like m:metrics. These give the best overview of what is being used today on networks, but you have to keep the stats secret :)
  2. Access public statistics from content portals (most don’t offer these). It is very important to note that portal statistics have built-in bias – they reflect what was downloaded, but that is shaped by what is available. Platforms which are easier to develop for can get flooded with software, and therefore generate more downloads, leaving other users out in the cold. Also some portals, for example the free content site GetJar (and associated stats site MobRef) tends to offer a lot of, ahem, pictorial content and so gets heavy hits from people with larger screens. As it were. There’s also a strong smartphone bias as people searching down that site are likely to have bought their smartphone specifically to run 3rd party software on it.
  3. Try to collate phone shipment statistics. These tell you relatively accurately how many devices each manufacturer is shipping, but you don’t tend to get any more granularity than that – and LG or Samsung in particular have a nearly incomprehensible platform strategy with many incompatible handsets. Also, many users do not upgrade every year so knowing how many handsets shipped this month, quarter or year does not actually give you a true picture of what is being used in the market today. Breakdowns by country are also near impossible to get, and you have to take what you can get with time periods because what you can find (without purchasing reports) won’t be what you need.

Often the best source of statistics is to just look at your weblogs though. This will tell you what types of people are trying to access your content, and over time will generate some more usable information.

Tracking Down Some Figures

For our latest website revision, I wanted a graph comparing the market penetration of the different software platforms, with figures that could be verified by anyone using public web resources I could link to. This proved difficult, but I’ll explain the process I went through below.

Shipment figures usually come out some time in arrears, so for this exercise we’ll look at shipments across the whole of 2006. A good starting point is Canalsys, who do quite detailed
breakdowns of the smartphone segment of the market – in 2006 smartphones accounted for only 64m (under 7%) of the 1bn handsets shipped, but platforms outside of Java are very tied to smartphones.

The Canalsys report immediately tells us the OS-based market shares: Windows Mobile accounted for 14% of smartphones (~9m), Linux was 6% (~4m) and Symbian totalled 67% (~43m). We can further breakdown Symbian – Nokia shipped 50% of smartphones and they (basically) only ship S60, which means 32m of those were S60 and 11m must be split between UIQ and the Japanese locked-down Symbian. The report reveals that Sony-Ericsson shipped nearly 1.2m devices, most of which were in Japan – these days Sony-Ericsson basically account for all UIQ shipments, so it should be safe to assume that at most 1m were UIQ and the other 10m are locked-down Japanese Symbian phones, not able to access 3rd party content. As a quick sanity check, I found that Symbian claim they have now shipped 20m phones in Japan and the second 10m took only a year, which tallies.

For Flash Lite stats, we can look to Adobe who are keen to buff up their credentials – Vision Mobile has done a nice summary of various Adobe sources here. First thing that struck me – Adobe predict 1bn devices cumulatively shipped by 2010; in contrast, Sun state 2.1bn Java handsets have already shipped and predict 1bn new devices will ship just next year. But I digress. Adobe/Vision Mobile are claiming 220m “Flash-Lite-enabled devices shipped by end of 2006 (includes mobile handsets, PDAs and consumer electronics)“. Two problems – we’re trying to compare like-for-like, so we just want 2006 shipments of mobile phones – ignoring unconnected PDAs and consumer electronics, and ignoring pre-2006. My gut also says that this may be a misquote, and it might mean 220m devices could run Flash Lite – it only really started to be embedded in handsets (as opposed to purchased/downloaded by users) in early 2006 outside of Japan, which had 20m Flash handsets at this point.

So what to do? Check out Adobe last financial briefing in detail – and it turns out you can guesstimate the shipments from the mobile PDF‘s graph on page 13 – by my count, 151m across 2006 (15% of all devices shipped), a massive increase on 2005. There’s no way to tell how many of these were mobile handsets, and furthermore which were like the Samsung D600 (Flash sometimes used for menus but not available to end users) and which actually had it in a usable form, so we’ll be charitable and assume all of them (this page gives some hints but no stats).

AJAX is available on mobile platforms inspires a huge amount of hype and very little hard fact. Firstly, you can get it in two main forms – with the Safari browser available on Nokia S60 3rd edition phones and the iPhone, and with Opera Mobile for Windows Mobile, S60 and UIQ ($30 to download, but shipped as standard on some devices). Opera Mini, the incredibly popular Java browser that runs across most Java handsets, does not feature AJAX (as of the current v4) – it does do some clever server stuff to simulate some AJAX features though. Many incorrect blog posts and articles have been written about this, and Opera keep their FAQs vague. There are some other attempts to bring AJAX and widgets to mobile but they aren’t simple adaptions fo AJAX ont eh web so we’ll ignore them here.

AJAX was added to Opera Mobile in early 2006 – Online Reporter presents a good overview of the key platforms and claims 17m AJAX-capable Opera browsers shipped that year. The iPhones is too recent to appear on our comparison (half a million shipments in the first week is impressive but actually negligible on the scale we’re working to); S60 3rd edition handsets started shipping in March 06 and the S60 blog claims that by mid-2007 they had shipped 50m units (possibly using some dubious maths), so lets split the difference and say there were 25m shipped in 2006. This gives us approximately 42m AJAX devices in 2006.

This leaves us with Java, which turns out to be the hardest. Sun probably know, but they aren’t telling. Stats for many years ago are easy enough to find, but no use – so I had to resort to guesswork. I started by looking at the handset shipments for the year and make estimates based on a gut feeling for what each manufacturer ships – the big 5 account for ~807m shipments in 2006; we know most Nokias/Motorolas (except for the emerging market devices) and almost every Sony-Ericsson will have Java; many Samsungs and a lot of LGs also have it (but in their home market they use Wipi and that accounts for maybe 75% of the 40m phones used there). Many of the “Others” also have Java, eg. most Windows devices (such as all those made by HTC,the biggest Windows white label manufacturer). We also know Sun reckon that 2007 will see nearly a billion Java phones shipped, so a figure round the 700-750m mark would seem quite reasonable in the absence of anything more reliable. I hate this kind of guesstimate but it’s currently the best I can come up with.

So we end up with a nice comparison graphic for handsets shipped in 2006:
Mobile Software Platform comparison graph

Some more up-to-date figures would have been nice but hopefully the protracted analysis made it obvious that it’s extremely difficult to reliably obtain even older statistics. It’s always worth remembering that these are only approximate figures anyway – broad trends can help you pick the right platform, and once your product is out that’s the best guide you can have.

Please comment on the original post.