Moving Towards Over-the-Air Updates

The XDAndroid Project currently distributes system updates as compressed, whole filesystem images which the users then unpack onto their SD cards, replacing the originals. This is done for the rootfs, initramfs and system images. Additionally, the project distributes updated versions of the (re)bootloader (HaRET) configuration files and the Linux kernel used to boot into Android.

By far the largest individual piece is the system image (system.ext2). This image typically runs around 60-70MB compressed. Updating in this fashion is not a terrible inconvenience for most users, however the files are large, require tedious uploading by the packagers (and more waiting time for the users) and suck up tons of bandwidth. This is problematic for the servers that host the images, the packagers that upload the images, and the users that download the images (particularly those downloading over the cellular network directly).

To alleviate this, I have been considering (for some time now) rolling over-the-air update packages in the same style as official Android updates from Google. These packages are zip files, signed by and authenticated as originating from the XDAndroid project. The zip files contain a series of binary patches used to update from an old build to a new build without replacing the entire system image.

Over-the-Air (OTA) update packages are useful for a number of reasons: 1. they’re much, much smaller than full filesystem images (24MB for the FRX01 to FRX02 OTA update file); 2. they’re guaranteed authentic (our installer only handles packages signed by the project); 3. they cost less for the project to distribute and the users to receive, in both data and storage space; etc. The only problem? We have no bootloader, which is what traditionally invokes the update process on a native Android device.

Well, problem solved. In the latest rootfs at the time of writing this, I have added over-the-air update support. Since we have no true bootloader, we rely on the rootfs to act as such. In a way, this works out to be a bit easier, since our rootfs has an entire embedded Linux environment to work with. All we need to do is check for an update.zip package and invoke the updater. Piece of cake!

From a user’s standpoint, the update process is done like so:

  1. An OTA update package (update.zip) is downloaded from XDAndroid’s servers
  2. The update.zip file is copied (NOT extracted) to whichever directory your startup.txt file is in (ie. /sdcard/andboot)
  3. The user boots HaRET and enters Linux
  4. The rootfs finds that update.zip and starts updating the device
  5. The updater completes (hopefully!) and reboots the device (into Windows Mobile)
  6. The user boots HaRET again and the device boots to the updated Android system.

Ultimately, this new OTA update support will eventually allow us to roll an Android program that will download these updates (and perhaps also kernels and rootfs and initramfs images) and place them on the SD card automatically. Until that’s ready, though, it’s just a matter of the user putting it on the card as update.zip. Still no more difficult than a system.ext2 replacement, while being much less taxing on our Internet pipes.

So there you have it: a nice, verbose description of a neat little new feature. We look forward to (probably) distributing FRX03 as an OTA update package (as well as a system.ext2 for those of you who like to do that). And finally, since I know everybody wants to see some proof, here’s a screenshot of the updater in action on a my RAPH110/ATT Fuze…

The XDAndroid Updater in action

XDAndroid 2.2.1 Build FRX02

In case you’ve missed it, the XDAndroid Project today released its newest Froyo series build, FRX02. This is the second of our official releases to have a build number. We’ve got some really nice bug fixes, improvements and features for you this time.

Here’s an abbreviated list of changes since FRX01:

  • The following bugs were fixed:
    • 2 – Talk.apk missing
    • 4 – Buttons cut off in the open call menu.
    • 14 – Startup.txt file is incorrect
    • 20 – OpenWnn IME selected by default
  • Google Apps updated to 20101020.1
  • Transitioned to hdpi graphics and fonts
  • Ambient light sensor and hardware auto brightness for RAPH and RHOD (WisTilt2)
  • Debug output for battery service emergency shutdowns (by request of camro)
  • Data roaming off by default (can be dangerous for international users) (emwe)
  • armv6j instruction support from cyanogenmod

I love the light sensor support that WisTilt2 came up with. Give him lots of thanks (and donations!) for the great work.

Check out the release from your preferred thread on XDA-Developers (I like the official Raphael thread myself) and give us some feedback on IRC or file some bugs! Thanks again for using XDAndroid!

Important XDAndroid Button Changes for Touch Pro/Diamond

For a long time, Raphael and Diamond users have had an… interesting button configuration, to put it nicely. New users had to learn how to ignore the labels on the buttons in order to effectively use their device.

With recent changes in the kernel to provide a different keycode for the Power button, I’ve had to reconfigure the HTC Fuze (RAPH110) layout to fix that button. At the same time, I decided it’s convenient to remap all the Raphael and Diamond buttons to their intuitive functions.

As of the next rootfs release, the device buttons will be performing the following functions…

Device Key    |   Android Function
----------------------------------
Power             Enter and exit sleep mode
Home              Go to home screen (long press, Recent Apps)
Back              Back (same as before)
DPad Center       Menu
Start Call        Dialer (same as before)
End Call          End call (same as before - configurable in Spare Parts)

This is only for Touch Pro and Touch Diamond users. Please see our tracking bug for the development discussion that led to this change.

XDAndroid Bug Tracker

This is old news for most users by now, but the XDAndroid project now has a bug tracking system. It’s a standard Bugzilla installation, so it will likely be familiar to many users.

This was done to facilitate better bug reporting by our faithful users, of course. However, it also is meant to help developers (me) keep track of what must be done for the next release.

Another nice side effect of the bug tracker is that it makes the developers’ work more transparent. Users can now see progress made on bugs in a more verbose manner (aside from the IRC channel). And for when we get close to a release, they can check up on bugs that are scheduled to be fixed in that release — and possibly help fix them or test their fixes. For instance, users can search for bugs that will be fixed in FRX02. Closer to release, we’ll have a release engineering bug which depends on all of those being fixed (for easier search accessibility).

So if you come across any issues in XDAndroid, check out the tracker. If you don’t see your bug there, open an account (we don’t spam!) and let us know about it. This tracker takes care of both the kernel and the Android filesystem images.

XDAndroid 2.2 Build FRX01

The XDAndroid Project is pleased to announce the first stable release of XDAndroid 2.2 (Android Froyo). This build is a significant milestone in the project. This is an important release of the system image component, which is just one piece of the bundles we release occasionally. Incremental releases of the other components will continue normally. Read on for a full, somewhat technical, list of important changes in this build.

Continue reading “XDAndroid 2.2 Build FRX01”

XDAndroid Shuts Down

That’s right, the XDAndroid system is no more… after you select Power Off from the power menu. With phhusson’s help, we found a magic switch that powers off the device. This is currently not well tested, but has worked for me on a RAPH110 (AT&T Fuze). We’ve had varying reports of success. In one incident, I had the device turn on automatically a couple hours after shutdown, so we still have to look at the shutdown procedure.

(To my followers freaking out about The XDAndroid Project “shutting down”, I sincerely apologize for the title of this post, but it’s too good to pass up.)

In other news, one of the fine folks over at the PPCGeeks forum discovered something interesting regarding battery life on our devices. Apparently some of the Touch Pro2 chargers have a little LED in the transformer block  which indicates a connection to the device. Interestingly, when the poster left the charger plugged into his device and removed it from the power outlet, the LED stayed lit. This suggested that there was power being drawn by the USB port, even when nothing was connected to it.

After a lot of mucking around with debug interfaces (which led to the power down thing from above being discovered), we finally tracked it down to the HTC Headset support (internally named HTC 2 Wire or H2W). After disabling the driver, the port was no longer active. In my internal testing, I found there was likely a small gain in battery life with the port disabled. Unfortunately, it seems like this difference isn’t very big as I was still able to burn through about 40% of the battery (in WinMo measurement) after under 7 hours.

We’re still tinkering with some battery related things, and trying to get a better understanding of what certain parts of the system do. Hopefully we’ll stumble across something useful like we did with the shutdown switch.

Thanks for reading!

Froyo Progress Update

Another week, another Froyo update. Before I begin: if you’ve been building systems, we now have updated build documentation on the XDAndroid wiki for the froyo branch. Check it out!

Work on Froyo has continued to progress rapidly. The bugs are starting to dwindle and I’m beginning to work on features more and more.

GSM users will be happy to hear that I’ve finally integrated the Access Point Names (APNs) list correctly, and it will be installed with babijoee’s next system release. CDMA users shouldn’t feel left out either, since with the help of hamagc on IRC, I was also able to add the CDMA generic APN again. The vast majority of providers should have settings that work out of the box from now on.

Many users reported shortcomings in the on-screen keyboard. Touch sensitivity is still a bit off (and I don’t see that changing — these screens are pretty small), but I was able to pull some autocomplete dictionaries out of cyanogenmod’s source tree. The dictionaries make the keyboard much more useful, of course. This will also be in babijoee’s next build.

Finally, by popular demand I have re-enabled the touch-friendly incoming call screen. This screen is similar to the unlock screen when you turn the device back on: it offers two pull switches to answer or reject an incoming call. The touch-friendly screen displayed during an accepted (or dialed) call, however, is still disabled. The layout is buggy on VGA phones.

I’ve begun investigating the possibility of issuing over-the-air updates for XDAndroid systems. Building OTA update packages is easy, thanks to the Android build system. The difficult parts are integrating an update mechanism into our rootfs, and developing infrastructure for Android’s checkin service to contact and check for or receive updates. This would be a pretty neat feature down the road.

Thanks for reading!

Froyo Progress Update

It’s been a while since the last XDAndroid update. We’ve since released several testing builds of Froyo, all of which have been progressively more useful and stable.

Since babijoee’s beta-2 release, the primary focus of development has been on bugfixes. Many users have been running the builds and offering great feedback on what needs work. With their help, I’ve been able to make some major fixes in the system.

First off, lots of people reported that their phones weren’t getting data connections out of the box. Users had to enter their provider’s mobile access point (APN) information by hand. This is an unacceptable issue and something that I missed due to testing without a SIM card in my device. The XDAndroid-specific bits in the source tree actually had a very large list of APNs in it, but it wasn’t getting copied over during system installation. This was a bug in our build system support introduced in Froyo, which has been fixed.

Second, the Google Apps support was suboptimal. The Google Apps packages we relied on include libraries that were built for newer devices which use different CPUs. Since our devices were not entirely compatible with libraries built for those CPUs, the applications were crashing on startup. There has since been a Google Apps package developed specifically for Froyo and devices with older CPUs. This package has been integrated into our build system. Specifically, this has fixed the Voice Search application and voice input with the Android keyboard.

Finally, I continue to make tweaks to increase performance and responsiveness of the user interface. With cues from cyanogenmod, I’ve made a couple of animations speed-ups and gave a hint to the system to use less sophisticated eyecandy features where possible. This helps improve the user experience quite a bit. There are still other places which can benefit from further optimization, so keep an eye out for more small performance gains in the future.

These changes have been added to the source tree, but are not present yet in current releases (as of 14 July, this post’s date). Development’s pace has been increasing and we still have some bugs to squish, so keep an eye out for more test builds. For users who wish to attempt to build their own system images, I have updated the build documentation on the wiki for Froyo’s new procedures.

PS: We still have not fixed the SD card bug noted by many HTC Diamond users, but I’m starting to get some great information from helpful users on #xdandroid.

Mario Marathon Prize Patrol

(Before I start, I’d like to apologize in advance to my XDAndroid followers: This post is cruft for you, so if you’re not into it, don’t feel bad to disregard it angrily. Also, for those of you who hate alliterations: I apologize for the title.)

For much of last week (and starting two weeks ago from Friday), the Super Mario Marathon guys ran their third annual event to raise money and awareness for the Child’s Play charity. If you haven’t heard of Child’s Play, they are a not-for-profit organization which raises money and purchases toys, games and books for children’s hospitals. The results of Child’s Play’s efforts have provided entertainment and raised morale for sick kids in hospitals across the country and even internationally. Child’s Play is a charity formed from the ground up by the gaming culture, and is one of the industry’s most significant contributions to society.

Read on for some thoughts about the Marathon’s incredible success! Continue reading “Mario Marathon Prize Patrol”

Fun with Froyo

Time for another quick update! Anybody reading this post has probably already noticed the prerelease of our new XDAndroid Froyo system image. Development on the new Froyo system is going very fast and quite smoothly. Certainly, our initial system image is not production quality yet, but we’ve already made great progress on fixing the bugs and restoring features.

Since the prerelease, we’ve managed to get a few features working on parity with our Eclair system (this is what you can look forward to in upcoming prereleases):

  • MicroSD card support is back
  • Bluetooth “works” again
  • ADB is working and we use our rootfs utilities on the shell
  • Countless small fixes

However, there is still a lot that needs work. We have a few notable issues…

  • WiFi still doesn’t work correctly
  • Some Google Apps are broken
  • MP3 playback doesn’t work
  • And many smaller fixes

There is ongoing development in the XDAndroid repositories on gitorious. Developers and advanced users are free to check out the source code via repo, but there is no build documentation yet. Once the Froyo AOSP source tree is relatively stable, we will be adding that information to the XDAndroid wiki.

PS: At the suggestion of IRC regular hamagc, I have created an XDAndroid Paypal account, where we will be accepting donations in the future. Please feel free to donate. Thanks!