Beyond Logic

SquashFS & LZMA Compression

    Prior to firmware version 62.51.1, the squash filesystem contained within the NB5 was compressed using the conventional ZLIB library. There has been a shift in recent times to LZMA (Lempel-Ziv-Markov chain-Algorithm) due to better compression size. There are some claims that LZMA can offer up to 30% better compression over ZLIB, however a 10 to 15% gain is normally achievable on the typical squashFS images found in modems such as the NB5. One of LZMAs disadvantages is it requires more memory and CPU to compress the data, however only uses 1/10 of the memory to decompress. This makes it somewhat ideal for embedded systems where the compression is performed once on a kernel image or filesystem using a desktop computer, and the decompression is done frequently on the embedded system with limited CPU and memory.

    There has been a lack of interest from the linux kernel maintainers to implement LZMA compression or decompression into the kernel, as they claim ZLib does just fine. As a consequence, SquashFS doesn't natively come with LZMA compression out of the box (or tarball). SquashFS using LZMA currently requires patches, and unfortunately there is a couple of versions floating around were the format of the headers/properties are different and hence cause incompatibility problems. The patches here, are designed to be compatible with the LZMA squashfs support found in the linux kernel of the Texas Instrument's NSP.

    Building mksquashfs

    The squashfs version found in the TI NSP MontaVista 2.4.17_mvl21 linux kernel is still version 1.x. Therefore if you intend to create a Squash filesystem compatible with the kernel in the NB5, you will need to use SquashFS v1.3r3. We have observed further size improvements of approximately 9 to 10% using the current SquashFS 2.2. Therefore the LZMA patches here have been designed to compile as a LZMA library which has the same function calls exported than ZLib. This allows any version of SquashFS to be easily linked to the LZMA library in the future.

    Download the LZMA SDK, patch and build the LZMA Library.

      mkdir lzma427
      cd lzma427
      tar -xjf ../lzma427.tar.bz2
      patch -p1 < ../lzma427_zlib.patch 
      cd SRC/7zip/Compress/LZMA_Lib

    Then download an appropriate version of SquashFS, patch and build.

      tar -xzf squashfs1.3r3.tar.gz
      cd squashfs1.3r3
      patch -p1 < ../squashfs1.3r3_lzma.patch
      cd squashfs-tools

    This should successfully build a mksquashfs tool that you can use to create LZMA Squash filesystems for use with the NB5 firmware version 62.51.1 and later.

    Adding LZMA Squashfs support to your development machine's kernel

    Now that you have a utility to make a LZMA tightly compressed squash filesystem, you will more than likely require LZMA support in your desktop kernel to open/modify existing filesystems or test your own. There is no requirement for a older version of SquashFS as support is backwards compatible. This patch will provide LZMA support to a linux-2.6.12 kernel that has squash2.2 installed.

    Mounting the NB5plus4 and NB5plus4W images

    The firmware for the NB5 appears to be distributed as three separate files, the kernel, squashfs & config.xml. However to assist users in upgrading the NB5plus4 and NB5plus4W units, a single firmware upgrade file is used with the three files concatenated together and a header attached to the front.

    To mount the squash filesystems from these images, you need to know where the squashfs starts in the file. The easiest way of doing this is looking at the address in the header at 0x80 or searching for the magic "hsqs". You can then use this offset to mount the filesystem. e.g. for the NB5plus4W firmware upgrade, use :

      mount -t squashfs 1.C07NCT6.8267A_050601.firmware.upgrade.img /mnt/squashfs -o loop,offset=0x78000

    and for the NB5plus4 upgrade use :

      mount -t squashfs 1.C08NCT3.8227A_050518.firmware.upgrade.img /mnt/squashfs -o loop,offset=0x77000

Copyright 2005 Craig Peacock - 3rd September 2005