Archive for Mobile

Mobile Webapps – iUI Framework Extensions

// August 3rd, 2009 // 10 Comments » // Mobile, Web

As I explained in my earlier Masabists post, I’m finally able to show off some of the work I’ve been doing recently on local mobile webapps.  We’ve based our webapps on the rather excellent iUI framework, which has a great philosophy:

  • Clean, standard HTML markup;
  • Very lightweight – one CSS stylesheet and a single Javascript file;
  • Built-in AJAX support, so form values can be submitted and HTML fragment responses pasted into the document structure;
  • iPhone native-style look and feel, including slick screen to screen transitions and screen rotation support.

The screens are all embedded inside a single HTML file, which has two advantages:

  1. You get very rapid movement around the app, with no need for a reliable network connection and none of the slowdown associated with downloading every page;
  2. You can cache the app using an HTML 5 manifest, so even when the user has no network signal they can access the site.

This is great as far as it goes, but there were a number of screen types not supported.  I have added an optional extension script, following the same philosophy, to add them – this ost is just a reference point to describe what the extensions do, when discussing merging them into the framework.

Structure

The extensions are in a seperate script and CSS file, which creates the window.iui_ext object, much like the core iUI framework itself.

To work, iUI has been extended in one key way – it now fires off Javascript events to screens, specifically:

  • onFocus() is called whenever a screen is made visible;
  • onBlur() is called whenever a screen is left.

Both events are called at the point where the link is clicked and the sliding transition begins.

Adding the events gives the developer a huge amount of extra scope for controlling the webapp, which is then used to create the additional webapps without breaking away from clean markup.

Form Handling

The iUI extensions automatically add focus and blur event handlers for every form, which store all field values in a name/value store whenever a form is left and restore the values whenever a form is visited again.  These are added inside the iUI code, and once finished they will call any explicitly defined onFocus="..." / onBlur="..." attributes within the markup.

The map follows the same basic semantics as the HTML 5 sessionStore object, which I wanted to use but sadly it isn’t implemented on the iPhone yet (the latest desktop Safari seems to have it though).  The methods are exposed through the iui_ext object, for example window.iui_ext.getItem('key').

The purpose of this store is to allow the movement of values between screens, which enables a number of tricks – by declaring inputs on seperate forms (displayed as seperate screens) with the same name attribute, you can make sure they always mirror each other’s values.  If for any reason you don’t want this behaviour for some fields, you can declare an onBlur event and remove them from the store.

Option Lists

I needed a form screen which listed a number of fields which could be changes, such as railcard type in the screenshot below, and by tapping on the field the user would move to a list of all the possible options from which they could select one.  A few examples of this UI pattern from my iPod Touch Settings app are:

  • Settings > Music > Audiobook Speed (list of 3 options)
  • Settings > Video > Start Playing (list of 2 options)
  • Settings > Safari > Accept Cookies

Visually, it looks like this:

iPhone option selection UI widget

I implemented in two stages. Firstly, I generated CSS to render a radio input (with label) as a block with the tick if it is selected, using a bit of custom Safari CSS (-khtml-appearance:none;) to suppress the native style of a checkbox – which interestingly changes from relatively nice on an iPod Touch v2 to quite nasty on an iPhone 3GS.

Next up, I tried to think of a natural way to express the options in HTML, and ended up with the idea of getting a script to rewrite any select in a form which has the CSS class of ‘panel’ into a seperate linked panel.  The markup you write looks like this:

<div class="row">
 <label for="buy_railcard">Railcard</label>
 <select id="buy_railcard" name="railcard" class="panel">
  <option value="rc_none" selected>None</option>
  <option value="rc_youth">Youth</option>
  <option value="rc_family">Family</option>
  <option value="rc_senior">Senior</option>
  <option value="rc_disabled">Disabled</option>
  <option value="rc_network">Network</option>
  <option value="rc_hm">HM Services</option>
 </select>
</div>

By default, it would render like this (taken from desktop Safari, just to make my life easier):

Selects as they would render in Safari without the script

The script rewrites the above HTML on the original form like this:

<div class="row">
 <a href="#select__buy_railcard">Railcard
  <input type="hidden" name="railcard" value="rc_none"/>
  <var id="railcard-rc_none" class="_lookup rc_none">None</var>
 </a>
</div>

Points to note:

  • The original form will submit in exactly the same way to the server
    • the new hidden field within the link has the same name as the select and the value of the selected option.
  • The textual label of the selected option is copied into the var tag, which actually visually shows the selected option to the user
    • the var is also given the option value as a css class, which allows us to do the funky icons on all labels if we want (using :before generated content, in this case).

The script also creates a new form.panel screen containing the options as a radio group, like this:

<form id="select__buy_railcard" class="panel" title="Railcard">
 <fieldset class="radiogroup">
  <div class="row">
   <label class="rc_none" for="buy_railcard_option_rc_none">None
    <input id="buy_railcard_option_rc_none" type="radio" value="rc_none" name="railcard"/>
   </label>
  </div>
  <div class="row">
   <label class="rc_youth" for="buy_railcard_option_rc_youth">Youth
    <input id="buy_railcard_option_rc_youth" type="radio" value="rc_youth" name="railcard"/>
   </label>
  </div>
  <div class="row">
   <label class="rc_family" for="buy_railcard_option_rc_family">Family
    <input id="buy_railcard_option_rc_family" type="radio" value="rc_family" name="railcard"/>
   </label>
  </div>
  <div class="row">
   <label class="rc_senior" for="buy_railcard_option_rc_senior">Senior
    <input id="buy_railcard_option_rc_senior" type="radio" value="rc_senior" name="railcard"/>
   </label>
  </div>
  <div class="row">
   <label class="rc_disabled" for="buy_railcard_option_rc_disabled">Disabled
    <input id="buy_railcard_option_rc_disabled" type="radio" value="rc_disabled" name="railcard"/>
   </label>
  </div>
  <div class="row">
   <label class="rc_network" for="buy_railcard_option_rc_network">Network
    <input id="buy_railcard_option_rc_network" type="radio" value="rc_network" name="railcard"/>
   </label>
  </div>
  <div class="row">
   <label class="rc_hm" for="buy_railcard_option_rc_hm">HM Services
    <input id="buy_railcard_option_rc_hm" type="radio" value="rc_hm" name="railcard"/>
   </label>
  </div>
  </fieldset>
</form>

Date Selection

One of the greatest things in HTML 5, to me, is also one of the simplest – the input tag has now been extended to allow all sorts of new types, such as dates, telephone numbers and even colours. The idea is that browsers will then handle these specially with funky calendar drop downs etc that curretly have to be implemented in Javascript – and on mobiles, they can use a more appropriate native solution optimised for the keypad, with address book access etc.

Obviously, no mobile browser has bothered to implement these – today you’ll have to look at Opera desktop if you want to see a browser starting to do it correctly.

In the absence of a native widget, I implemented a date selector myself.  On the iPhone itself, this is implemented in a style similar to HTML drop-downs in Safari – difficult to achieve in HTML alone, and not actually that natural for selecting travel dates.  I opted to support date selection with a calendar reminiscent of theiPhone calendar app’s instead:

Date selection with iUI extension

As before, this is implemented with a standard piece of HTML that is rewritten by the script.  To add a calendar you add an input like this to your form:

<div class="row">
 <label for="buy_travelDate">Outbound Date</label>
 <input name="travelDate" id="buy_travelDate" type="date" class="date" value="1/1/2009">
</div>

The type you choose to assign can be either date or text.  If you set the input’s type to text by hand, the script will always replace it with a custom calendar.  Currently, Safari rewrites any type it doesn’t understand to text when the page loads; if and when date support is added, this will presumably cease and the script will stop handling dates for you and leave selection to the browser.  Your choice which you want!

When the form is first loaded, your markujp is rewritten like this:

<div class="row">
 <a href="#select__buy_journeytype">Journey Type
  <input type="hidden" name="journeytype" value="tt_single"/>
  <var id="journeytype-tt_single" class="_lookup tt_single">Single</var>
 </a>
</div>

There will also be a (singleton) extra screen created for the date picker like this:

<form id="_datepicker" class="panel">
 <p>
  <span id="_dpback" class="back"> </span>
  <span id="_dpmonth" class="month"/>
  <input id="_dphidden" type="hidden"/>
  <span id="_dpfwd" class="fwd"> </span>
 </p>
 <table id="_dptable" cellspacing="0" cellpadding="0" border="0">
  <colgroup>
   <col class="sun"/>
   <col class="mon"/>
   <col class="tue"/>
   <col class="wed"/>
   <col class="thu"/>
   <col class="fri"/>
   <col class="sat"/>
  </colgroup>
  <thead>
   <tr class="days">
    <th>Sun</th>
    <th>Mon</th>
    <th>Tue</th>
    <th>Wed</th>
    <th>Thu</th>
    <th>Fri</th>
    <th>Sat</th>
   </tr>
  </thead>
  <tbody>
   <tr class="wk1">
    <td> </td>
    <!-- etc -->
   </tr>
   <tr class="wk2">...</tr>
   <tr class="wk3">...</tr>
   <tr class="wk4">...</tr>
   <tr class="wk5">...</tr>
   <tr class="wk6">...</tr>
  </tbody>
 </table>
</form>

The screen is a singleton, which means it will only be created once, and reused for every date picker in your app (which saves on memory).  Month and day names are taken from arrays which could be externalised and internationalised easily if required (CSS class names on cols are hardcoded though).

Every time the date picker is selected, the script refreshes the calendar, working out the date from the hidden field on the original form and copying over the relevant details.  Table cell contents are updated with the relevant date, and onClick events set to make selections work.

There are a few options, which can be passed in using CSS classes applied to the date field:

  1. If the today class is specified, then any selection of today’s date will be stored on the form (and shown visually) as ‘Today’ instead of the date;
  2. If the future class is specified, only today and future dates are visible and selectable;
  3. If the past class is specified, only today and past dates are visible and selectable;
  4. A date pattern can be specified using the following syntax:
    • Parts of the date:
      • d for the date eg. 28
      • D for the day’s abbreviated name eg. Mon or Fri
      • m for the month’s number eg. 1 (for January)
      • M for the month’s abbreviated name eg. Jan or Dec
      • y for the year as two digits eg. 09
      • Y for the year as four digits eg. 2009
    • Other symbols:
      • _ underscore is converted to a space
      • c is converted to a comma
      • Hyphens and slashes are also allowed.
    • Examples:
      • d/m/y goes to 3/8/09
      • d-M-y goes to 3-Aug-09
      • Dc_d_M_Y goes to Mon, 3 Aug 2009

If a date format is set, then only that format can be used to specify the value of the field.

That’s basically it for my extensions – having documented them, I’ll now see if anyone wants them merged in to the main iUI framework!

Masabists: Ticketing via Local Webapps

// July 31st, 2009 // Comments Off // Mobile

This was originally posted on the Masabists blog.

We’re finally able to show off some of the work we’ve been doing recently on local mobile webapps – interactive web pages which can be saved and run even when you’re offline.

Our mobile ticket sales app is now available as a local Java app for mass market handsets, and a local webapp for Android and iPhone – offering all the same functionality, security and slick branding:

Masabi Train Ticketing local webapp on an iPhone Masabi Train Ticketing Java app running on a Nokia N96

What Is A Local Webapp?

With the latest HTML 5 and Google Gears APIs support on Android, iPhone and Palm Pre, you can provide a fast multi-screen interactive app with local storage (to store tickets you have purchased), which behaves like a native local app and is accessible even when the phone is offline. Here’s how to store the app for later use on an iPhone – reached by clicking on the ‘+’ icon in the footer:

Why Webapps?

Traditionally at Masabi we have always written local apps in Java, because they offer the best mass market user experience for the sort of ticketing and financial services we provide; it’s only with the advent of handsets with fast, HTML 5-capable browsers that we have been able to explore the webapp route. We plan to use local webapps for many of our iPhone and Android products for two core reasons:

  1. With the proliferation of new platforms causing even more fragmentation in the mobile apps space, the Safari browser used on both platforms is actually the safest way to reduce fragmentation and streamline maintenance and development;
  2. There are some big advantages to the unrestricted installation of a webapp via the web, especially if your business model is not compatible with the rules or revenue shares of the relevant App Store; it can be hard to justify to a customer the extra expense of a dedicated app when you cannot guarantee the app will ever be allowed on the store or device.

Fortunately, it is easy to take a local webapp and wrap it up as a native iPhone or Android app, so we can make all of the services available through the relevant App Stores as well if that is what the users want.

It is worth noting that at Masabi we are producing free mass market services which pay for themselves through transaction fees, so the App Store’s billing system isn’t an issue for us – your mileage may vary…

Does It Feel Like A Normal App?

Webapps can very successfully replicate the look and feel of native apps, with quick scrolling between screens, button styles and the like.
Below are screenshots of the user selecting an option from a list, and a date from the calendar:

Selecting an option from a list on the Masabi Train Ticketing local webapp on an iPhone Picking a date on the Masabi Train Ticketing local webapp on an iPhone

These clearly follow the style and usability conventions of the built-in iPhone apps. With CSS targetted to the device through our DeployME server, we reskin the same application easily to adopt Android conventions and styling as well.

Please comment on the original post.

Masabi StreetVendor mPayments app announced

// July 30th, 2009 // No Comments » // Mobile

My company, Masabi, have just announced our mPayments platform for the developing world – quite funky stuff which breaks the traditional need for operator involvement to handle the sorts of peer-to-peer payments currently transforming Africa and Asia.  You can read more about it on the release, or just listen to Ben demo and explain it all on YouTube.

It’s amazing how slowly news lags development – the app has actually been running in Sudan for some time, and is going to be deployed in other parts of the world soon (details under wraps sadly…).

Screenshot of the Masabi StreetVendor mPayment application running in Arabic - click to see more on Flickr Screenshot of the Masabi StreetVendor mPayment application running in Arabic - click to see more on Flickr

Great work from everyone involved!

Events: Global Messaging 2009 – Mobile Ticketing and Payments

// June 24th, 2009 // No Comments » // Mobile

I’m just back from presenting at the excellent Global Messaging 2009 event in Westminster, on the subject of what makes a good mobile service in the context of mobile ticketing and payments:

It covers some of our standard reference points, like mass market handset support and the need for a mobile service to address a user pain point and actually perform better than the alternatives to be used. I think this really applies to selling train tickets from the handset, something that a pure deliver-only model fails to achieve.

Sadly I fluffed the Sir Mixalot slide, slipping and double pressing the right arrow so we skipped past it.  The actual line I was supposed to say was “But – and it’s a big but – are we addressing the customer’s needs with a pure mobile ticket delivery solution?”. Which may not have made much sense to this audience anyway, so maybe it’s for the best…

Masabists: Sony-Ericsson Handset Announcements

// June 1st, 2009 // Comments Off // Mobile

Note: this was first posted on the Masabists blog and featured on Carnival of the Mobilists 177.

There are a few interesting points raised in the technical details accompanying Sony-Ericsson’s recent announcement of the Aino and Yari handsets – app usability should improve, whilst Sony-Ericsson’s smartphone platform strategy is tending towards the Samsung “try ‘em all” approach…

Concessions to Usability

Most importantly for developers, the new handsets aim to reduce the permission dialogue hassle that users experience when running applications on their handsets. This is a huge bugbear which just adds user pain without really improving security – because users rarely read the (often opaque) language of the message, they just click and get annoyed at the jarring interruption in their workflow:

MIDP permissions dialogue

The absence of these interrupting dialogues in iPhone applications adds to their usability.  They have clear purpose, but MIDP has always erred too far to the side of caution, and allowed explanation messages to be written and translated by programmers not usability experts, to confusing effect. They simply annoy, and give the platform a slightly amateur feel.

Sony-Ericsson suggest that Unsigned apps will now show far fewer dialogues, and Trusted 3rd Party signed apps will be able to ask once and then never bother the user again.  This is absolutely excellent news because it eliminates a key barrier to regular app usage.

Sony-Ericsson says this is in line with recommendations in the MSA spec – an umbrella JSR mandating the support for a huge range of APIs adding up to a very powerful platform, which is now gaining a lot of traction. This is also a great thing, because it implies that Sun recognise the problem, and that other handset manufacturers may be following suit.

Whilst in general I am in favour of reudcing the permission overhead to a secure minimum, this can go too far.  I am quite worried about the idea that dialogues may not be required when implementing the upcoming Javascript APIs designed to allow webapps greater access to the handset hardware – the potential for malicious scripting is huge.

Hopefully Javascript will find a more sensible balance where MIDP at first did not, because HTML 5 is opening up some exciting opportunities. Given, however, that browsers shipping today appear to be replicating all of the proprietary manufacturer-specific API problems of early MIDP, I’m certainly not holding my breath!

Post-UIQ Smartphone OS Standardisation?

In the good old days, Sony-Ericsson were pioneers in the smartphone market with the P800, back when everyone thought a smartphone should be like a PDA.  Looking back, my gut feeling is that on every subsequent revision UIQ (a UI layer built on Symbian) dropped further and further from the leading edge, striking an unhappy balance between mainstream usability and PDA power. UIQ finally went under when Nokia took Symbian open source without them, leaving Sony-Ericsson without a smartphone platform.

This is roughly the point when Sony-Ericsson launched the Xperia, a Windows Mobile handset that was very interesting, though ultimately didn’t fully live up to expectations.  A lot of effort was put into encouraging developers to build panels and things (proprietary UI extensions), but to the best of my knowledge they haven’t announced any more Windows Mobile devices – and hopefully they’ll stay that way.

Subsequently, Sony-Ericsson stated they would be focussing on the Android platform for future smartphones… and then announced two S60 handsets.  They have dropped hints that Android wasn’t to displace other OS platforms, but can a company in Sony-Ericsson’s state really support simultaneous development on two or three smartphone OSs as well as their (actually very good) proprietary feature phone OS?  Note that smartphones are only around 13.5% of current world handset sales (although at the lucrative end)…

Please comment on the original post.

Welcome to MoMos Jo’burg, Malmö, Oulu and Edinburgh!

// May 25th, 2009 // No Comments » // Mobile

Four new Mobile Monday chapters have been added to the global family over the last few months, in Jo’burg, Malmö, Oulu and Edinburgh.

I created a map some time back for our MoMo Estonia chapter site as the easiest way to keep up-to-date with the rapid growth of MoMo (other chapters – feel free to embed it on your web sites).  With the new chapters added – including the northernmost (Oulu) and the first in Africa (Jo’burg) – it’s now looking very funky:


View MoMo Chapters in a larger map

At some point I need to write some Javascript that can pull the KML location feed out of the map and generate nice little chapter lists, which some chapters have statically embedded on their sites right now (a nightmare to keep up-to-date, hence the map…)

Masabists: Interesting Blackberry Factoids for Developers

// April 22nd, 2009 // Comments Off // Mobile

Note: this post was originally written for the Masabists blog.

We’ve just had a great chat with the extremely helpful RIM Developer Relations guys, and some things came out of it which we were only slightly aware of. None are secret, just buried in spec documents and forums, so I thought it would be worth putting them up on the blog in case they are helpful to other developers:

Applications Permission Manager

Since Firmware 4.2.1, there has been a manager object which can not only tell you what the current API permissions are for your app, but allows you to display the device’s application permissions dialogue with your favoured settings pre-populated, which the user can then choose to save if they are comfortable with them.

Used correctly, this is a great way for the app to discretely flag how the user can make their life easier, without being too intrusive. We’re all about removing pain in a secure fashion, and love this! There are some pointers in this presentation, and from there you can dig into the relevant API.

Device Upgrades can Migrate Applications

A user can migrate their applications to a new device alongside their contacts, mails etc. On the face of things this is very useful, but it means your app’s cod files must be as happy working on a scrollwheel Suretype device with a 240×260 screen (eg. 7100) as they are on trackball QWERTY devices with 480×320 screens (eg. a Bold).

This could have real implications for games, which often use large graphics scaled and optimised for a particular screen size. It also impacts some strategies for providing UI movement and widgets across different keyboard and scroll/pointer types on fully branded Canvas-based applications.

The application’s RMS memory migrates with the app, and applications can query the device to find out its unique ID and specification, so if this is a huge problem you should always compare IDs at startup and issue an upgrade so that the user picks up a more appropriate cod.

The Storm’s Touch Screen Prefers Native Applications to MIDlets

If you want to provide proper support for the Storm’s touch UI, you’ll have to do it with a native UIApplication instead of a MIDlet. The MIDP Canvas touchscreen API is not expressive enough to handle the two levels of click used on the Storm.

Newer Blackberries Can Read Custom Jad Parameters

A huge bugbear with Blackberries before was their inability to read any parameter from the jad – this made the provisioning of unique information at installation time all but impossible. The MIDlet.getAppParameter() method simply returns any value stored in the Manifest at compile time, but nothing added in the jad afterwards.

Since firmware 4.3.0, this now works – and RIM are very good at providing clear User Agents identifying the device and its firmware when browsing (unlike some others), so you can make this call at provisioning time.

There’s also a great tutorial on Blackberry vs. MIDP code signing (PDF). I hope some of that information is of use to people – the guys from RIM were extremely helpful and we certainly were very happily surprised how much we got out of the meeting.

On an unrelated side-note, we’ve started Twittering – we’re just restricting the company tweets to information relevant to mobile applications, security and transport so if any of that is you’re bag feel free to follow us!

Please comment on the original post.

MWC 2009 talk: Challenges Building Secure Mobile Applications

// February 26th, 2009 // Comments Off // Mobile

Mobile World Congress 2009

Note: this post was originally written for the Masabists blog.

Ben Whitaker and myself gave a presentation in the App Garage of Mobile World Congress in Barcelona this year. The slides of the talk are reproduced here – let us know if you’d like the full code samples as well. Apologies for the front page, which seems to disagree with the Slideshare conversion system…

Please comment on the original post.

Pictures from Barcelona

Photos I took whilst in Barcelona for MWC:

Traffic Light Spinvox Men MWC-30 Barcelona site seeing Santa Maria del Mar Cathedral Sagrada Familia Sagrada Familia door Windows MoMo Peer Awards Spotlights Port Vell Curves Door Chimney

Masabists: Transcoders round 2

// February 13th, 2009 // Comments Off // Mobile

Note: this post was originally written for the Masabists blog, and was nominated Post of the Week on Carnival of the Mobilists #161.

It has been incredibly busy pre-MWC and so I have not yet pulled together our research into mobile browser handling of SSL certificates (as promised when I described transcoders). However one piece of news inspired me to write an addendum – Novarra, a transcoder vendor, have now started offering a version of their transcoder for laptops using 3G dongles. Bill Ray isn’t certain about all the details, and Novarra still haven’t replied to my previous request for information so it is difficult to confirm anything, but I’ll restate the problem now and draw some possible conclusions from that.

The problem as it stands: transcoder vendors want to rewrite content to improve performance and accessibility of sites accessed through “mobile” connections. To do this for secure HTTPS sites, they insert themselves into the transaction quietly, breaking end-to-end encryption and creating a potential security hole – though if they are doing their job correctly it should be hard to exploit. More details here.

The transcoder vendors are asking the W3C to approve a new best practices guide which states that they will be allowed to insert themselves into an HTTPS connection, but if the server has set a special HTTP header they will immediately stop and allow end-to-end encryption to resume. If you’re interested in the politics WapReview have an excellent post covering them.

On the face of things the proposal sounds eminently reasonable. It isn’t.

One year ago Netcraft reported that there were 2,451,780 sites on the web responding to SSL (HTTPS) requests, 794,008 of which certificates verified by trusted third parties such as Verisign. Growth was nearly 40% year-on-year at that point, so we can assume there are at least 3m sites today, 1m of which are serious about their security.

There are approximately 5-15 transcoder vendors (that’s an educated guess – there are more than two and I can’t imagine there are as many as 20).

The vendors believe that for security to be guaranteed, the best thing to do is that they break the existing standard, and all 1m (or 3m) sites on the web must change their server configuration if they want to continue being secure. This agreement will be made by a committee of a few dozen people at W3C – no word yet on how widely it will be publicised, but the people publicising the process so far are largely outsiders who don’t like the sound of it.
Many might not care, because they don’t support mobile handsets directly and don’t plan to in the near future. The move to broaden transcoding into laptop connections may make those people think twice, especially when you consider that in the future WiMax and LTE will start to offer true broadband over the air and the entire nature of the internet connection industry will change and become more wireless.

It seems common sense to a layman: the 3+ million e-commerce sites, corporate intranets etc should be allowed to remain secure without having to change all of their configuration; the dozen or so transcoder vendors should honour the existing system. If it is neccessary to transcode HTTPS connections, this should be an opt-in service decided by each and every site, who should be given full explanations of the (minimal?) risks – because they have the burden of care to their users, adherance to banking/privacy regulations, etc.

Let’s hope the W3C make the right decision, but if you are concerned about this maybe it’s worth publicising it widely just in case?

Please comment on the original post.

Masabists: How Do Transcoders Affect HTTPS?

// January 23rd, 2009 // Comments Off // Mobile

Note: this post was originally written for the Masabists blog, and was featured on Carnival of the Mobilists #158.

After reading the interesting discussion about mobile transcoders and HTTPS security on the MoMo London mailing list (may require sign-in), based on some related discussion around the new W3C guidelines for transcoding, I thought there was some value in a blog post which explains the issues from more of a layman’s perspective.

First up – transcoders are servers which sit between a mobile phone and a web site, and reformat the web site to (hopefully) make it easier to navigate and use on a handset. A number of operators have installed them, and their customers generally not aware of this – hopefully they just receive a better user experience. This is a subject with a chequered past, that arouses considerable passion among those for and against – I’ll take a neutral stance in this blog post and purely discuss the security implications of what is going on!

In a conventional HTTPS connection, as made by a desktop browser, the browser makes an “unbreakable” connection directly to the web site you are accessing – anyone between you and the site can view the bytes passing between the two, but cannot understand what they mean. Note that, for our purposes here, an “unbreakable” connection is one which is impractical to break within weeks using currently available technology assuming no compromise of the server’s private key – in reality nothing is completely unbreakable! The same should hold true for an HTTPS Wap2 connection between a phone and a web site, without any transcoders:

If you double click on the little padlock on your browser when accessing a site over HTTPS, you’ll see a certificate from a trusted certificate issuer (eg. Verisign) stating which domain you have the end-to-end connection with:

On phones, you should be able to do the same thing through the menu system – here’s one on Nokia S40:

Alarmingly, on this same Nokia S40 browser, the Organisational Unit label of the certificate is displayed instead of the Organisation name if both are specified, which often has no meaning to the user. For example Barclays Bank does an excellent impression of being a fake certificate to users not in the know:

However, in principle and when well implemented, the certificate system allows users to trust they are talking to the correct server with some rigorous maths in the background confirming everything. Desktop browsers are increasingly finding ways to automate additional certificate checks and flag up anything unusual, and the mobile web (with its more fiddly certificate checking) will doubtless follow along slowly.

It is worth noting that older Wap1 “secure” connections were not really secure. The handset communicated to the operator’s gateway over a WLTS connection, which was allowed in most cases to use lower key strengths and weaker algorithms which could be broken more easily. The gateway server on the edge of the operator’s network would then decrypt the information and send it on to the end web server over a true HTTPS connection. There were therefore two weaknesses – interception between handset and gateway, and hacking of the gateway to view the connection in plaintext:

These days almost all handsets use Wap2 – which allows for true end-to-end HTTPS. However, as an aside you can easily skip – and here I hold my hands up and declare I know nothing definitively – it is ambiguous what happens when a Wap2 handset connects using Wap network settings. Wap2 is allowed to work just like Wap1 through a gateway, but also prefers end-to-end TCP/IP that is bridged through the gateway but does not need extra translation done there, conventionally configured using “Internet” network settings on the handset. Most operators also provision “Wap” settings on handsets, which were traditionally used to connect an old Wap1 browser to the gateway using a different protocol which emulated some aspects of normal web protocols. When a Wap2 browser (that is also able to handle Wap1, as most are) tries to connect to a server over HTTPS through a Wap connection, is it using weak WTLS to the gateway as in an old Wap1 connection, or is it still using a true HTTPS connection? I am not certain of the answer, and I imagine it would depend on the handset model, browser software and network’s setup (as some gateways can auto-switch between both transparently) so it is difficult to test definitively. If anyone knows please clarify in the comments!

Back to transcoders. The controversial aspect is the way that a transcoder inserts itself into this secure connection in order to “improve” the page markup (I’ll leave the argument about whether transcoders do or don’t improve the markup to others). The transcoder forces the browser to make its secure connection to the transcoder, which then makes a subsequent onward connection to the real web server – a model which looks remarkably like Wap1, with better security between handset and intermediary:

Click on that browser padlock while using this scenario, and you would see the transcoder’s certificate and not the certificate of the site you are connecting to. The end result is: whereas in a conventional HTTPS connection, you only share your private information with the site you have chosen, in the transcoding case you share it with the target site and any employee of the transcoder company who has access to the transcoding servers, plus anyone who has hacked the transcoder server.
In reality, any responsible transcoder company would operate their servers with PCI-DSS compliance (the standard required of any company which takes credit card payments), and this would be a minimal risk, but it is more risk than if they weren’t in the loop.

I have discussed this with Novarra (who run Vodafone’s transcoder service in the UK) and they confirmed that they do operate their servers to these standards, but I haven’t talked to any others – and I think some of the concern people feel is that this is not an opt-in service, it is something done to your “secure” connection without your knowledge.
As an aside, Opera have long operated this sort of system with their Mini browser, and their FAQ makes it clear that you should not use the browser for any confidential connections such as accessing your bank – however Barclays Bank, for example, have long recommended Opera Mini for customers wishing to access their accounts on a mobile (recently they have added a disclaimer stating they have no responsibility if you do this):

There is a second problem with any system – Wap1 or transcoders – that obscures the certificate of the end web server. One of the key advantages of HTTPS is that, by providing end-to-end security and trusted certificates, you always know who you are talking to. A major class of attack is the “man in the middle” attack, where a server sits between your browser and the end site and takes a copy of all confidential information passing between them. If a server tried to do this with a true HTTPS connection you would be able to tell, because the certificate you saw would not match the server you thought you were connecting to:

If there was a thief’s server pretending to be the web server, it would not have the correct certificate and you could tell an attack was taking place:

A transcoder in the middle prevents this, because the end user only ever sees the transcoder’s certificate – the end web server’s certificate is only seen by the transcoder. As the user knows nothing about the features of the transcoding software, and the specific deployment configuration of that transcoder on their operator, they cannot tell whether the transcoder has been implemented to correctly verify the end certificate against what the user was expecting:

A thief could easily sit on the other side of the transcoder, and the user would know no better:

This is clearly a retrograde step – moving back towards the insecurity and uncertainty of Wap1. The worry is, if breaking HTTPS security becomes “allowable”, we are setting a very worrying precedent for trust in HTTPS and the security of mobile web commerce in the future. History shows that thieves and hackers are becoming rapidly more sophisticated with no signs of stopping – it is difficult to argue that, under those circumstances, we should make things easier for them.

Transcoding servers may take steps to confirm the identity of downstream web server’s certificates, and they may not (I am waiting for some confirmation on this from Novarra, for example, and will update the post if and when they provide it). They may prevent the user from accessing sites which have incorrect certificates, or allow the user to choose whether to continue. They may have the ability to do this but allow operators to make these policy decisions, who may not understand the implications. Already we know that some transcoders will not break HTTPS for some banks, but that would be no consolation to a corporate IT department who may unwittingly open up a potential hole in their corporate network, for example.

This is a company blog so I have to mention – the lack of certainty and lack of security on mobile was a key motivator behind Masabi’s creation of EncryptME, which powers our secure applications. We always provide our own full end-to-end security to wrap up transactions, either over the top of SMS or HTTP, giving true desktop-strength security from the mobile, regardless of what the operator or transcoder attempts to do in the middle. Because we understand security and have complete control over application workflow, we also know when to use more than just end-to-end encryption, which is a component of system security but never in itself a guarantor of full security.

Please comment on the original post.