Sunday, June 26, 2011

Ubuntu switched to Hybrid Disc Images: What this means for the End User

By default, Ubuntu 11.10 will be shipped in hybrid image format, to make it much easier to create bootable USB drives intended for both installing Ubuntu and using it as LiveUSB. In fact, the daily Oneiric images have already been converted to hybrid starting from July 16th. Both i386 and amd64 images are available as hybrid images.
  1. Developer's Statement
  2. What is a Hybrid Image?
  3. How to create a startup USB from a Hybrid Image?
  4. Adding the remaining space to a new partition
  5. What about the existing tools?

Developer's Statement

Colin Watson from Canonical announced on July 15th:

As of tomorrow's daily builds, all Oneiric amd64 and i386 CD images on can also be written directly to a USB device. You can still use usb-creator if you need to enable persistent storage on the USB stick, but if all you need is to install from the stick then this simplifies the process.

There is a small size cost to this feature; part of this is partition alignment to 1MB "cylinders" (since the image now has to simultaneously look like a CD and a partitioned USB disk), and part of it is a second directory tree whose size is roughly proportional to the number of files in the image. On the server CD (and probably the alternate CD too, though I haven't measured that), this came out to about 5MB, but as luck would have it I found a way to save about the same amount by hardlinking some files together, so the size there stays roughly the same.

On the desktop CD, unfortunately, we lose about 1MB (although I consider myself to have "paid" for this by way of the 2MB or so we gained by switching to live-build ...). If this becomes a problem close to release time,  then we may be able to shave off a bit at the possible cost of some USB-booting compatibility on a few machines by dropping the 1MB alignment.

I realise that we're behind a number of other major distributions on this. The delay was because (like Debian) we couldn't simply use isohybrid because that would break jigdo downloads, so we had to switch to xorriso as the CD image generator on these architectures for its new JTE support, and by the time all that landed in Debian I didn't really want to cram it into 11.04.

What is a Hybrid Image?

We are used to downloading ISO images from Ubuntu mirrors whenever a new release comes out. At the same time when these images can be burnt directly to a CD-ROM disc, for creating a startup installation USB, one needs to use specific USB creation tools like 'Startup Disk Creator' (included by default in Ubuntu), 'UNetbootin', etc. With hybrid images, you can simply write the data -including the filesystem- of the image to a USB drive by using the 'dd' command. Notice that just copying its content, after mounting it, won't work, as it wouldn't copy the MBR and the partition table, as well as additional sectors at the end of the image.

A major downside you have to consider is that this method will delete all data that was on the USB drive previously!

After writing the image on the USB drive, you will have the remainder of it as unallocated space, and you can create a new partition and use it for storing your files on the drive as well, while at the same time you can use it as the install/live medium. Notice that this will not create a persistent drive. See the section "Adding the left out space to a new partition" near the bottom of this post. Also, if you want to create a persistent drive, see the most lower section.

An advantage to note is that you'll have the real Ubuntu boot process (main boot options page etc.) as you have it when you boot a CD-ROM disc instead of the boot options you have when a UNetbootin created USB is booted.

How to create a startup USB from a Hybrid Image

Linux Command-Line

It is really simple. Go to a Terminal and type:

sudo dd if=~/Desktop/oneiric-desktop-i386.iso of=/dev/sdx oflag=direct  =1048576

Where '~/Desktop/oneiric-desktop-i386.iso' is the name and location of your downloaded image (located at the desktop in this example) and '/dev/sdx' is the target USB drive. If your system doesn't support 'oflag=direct', you can just leave it out as it is simply intended to speed up the process a bit.

If you don't know about the target USB drive path, run this command and figure out your destination drive.

sudo fdisk -l

Warning: Make sure to set the correct device path, as this process will delete all data that was on the specified device previously!

Remember, don't include an integer for the USB drive, e.g. '/dev/sdx1', as it would refer to the existing partition on that drive and not the drive itself.

When the USB has been properly created by 'dd', there should be an output similar to this:

malik@Natty:~$ sudo dd if=~/Desktop/oneiric-desktop-i386.iso of=/dev/sdb oflag=direct bs=1048576
706+1 records in
706+1 records out
740601856 bytes (741 MB) copied, 91.7024 s, 8.1 MB/s

Linux GUI

If you are still inclined to use a graphical interface for burning the hybrid image to your USB drive, you can try 'usb-imagewriter' from the repositories. Go to a Terminal and run this command:

sudo apt-get install usb-imagewriter

It would appear under Applications > Accessories > Terminal in classic GNOME or if using Unity, you can search the Dash for 'imagewriter'.

You would need to rename your downloaded image from 'whatever.iso' to 'whatever.img' as it only seems to support .img files.

Windows GUI

From Windows, you can use 'Image Writer', an open source project from Launchpad.

You would need to rename your downloaded image from 'whatever.iso' to 'whatever.img' as this also seems to support .img files only. You might need to disable the setting 'Hide extensions for known file types' under 'Folder Options'.

For clarity, click the 'Write' button to start the copy process.

Adding the remaining space to a new partition

Copying the image by following any of the methods described above would result in a new partition being created on your USB drive and that would be nearly the same size as the downloaded image. For the remaining space, you can use GParted to create a new partition that can be used for storing personal data etc.
  1. Install GParted if not already installed:
    sudo apt-get install gparted
  2. Start it from System > Administration > GParted or by searching the Dash in Unity.
  3. Choose your respective drive from the drop-down menu in the top right corner.
  4. Right-click the 'unallocated' space and choose 'New'.
  5. Choose 'Fat32' under 'File system' if you want to use the drive between different operating systems, or you can choose any other filesystem like Ext3, Ext2 or NTFS.
  6. You may also want to set a 'Label' (name) for the new partition.
  7. Click the Apply button in the tool-bar.

What about the existing tools, e.g. Startup Disk Creator and UNetbootin

Those existing tools are still compatible with these new hybrid images. In fact, if you wish to enable persistency on your drives, you need to use those tools instead of the ones described in this post. See here on how to create a bootable persistent USB drive using Startup Disk Creator.

Related Posts:

Installation , USB