SQUASHFS 4.4 - A squashed read-only filesystem for Linux Copyright 2002-2019 Phillip Lougher Released under the GPL licence (version 2 or later). Welcome to Squashfs 4.4. This is the first release in over 5 years, and there are substantial improvements: reproducible builds, new compressors, CVE fixes, security hardening and new options for Mksquashfs/Unsquashfs. Please see the INSTALL file for instructions on installing the tools, and the USAGE file for documentation on how to use the tools. Summary of changes (and sections below) --------------------------------------- 1. Mksquashfs now generates reproducible images by default. Mkfs time and file timestamps can also be specified. 2. Support for the Zstandard (ZSTD) compression algorithm has been added. 3. Pseudo files now support symbolic links. 4. CVE-2015-4645 and CVE-2015-4646 have been fixed. 5. Unsquashfs has been further hardened against corrupted filestems. 6. Unsquashfs is now more strict about error handling. 7. Miscellaneous new options and major bug fixes for Mksquashfs. 8. Miscellaneous new options and major bug fixes for Unsquashfs. 9. Squashfs-tools 4.4 is compatible with all earlier 4.x filesystems and releases. 1. Introducing reproducible builds ---------------------------------- Ever since Mksquashfs was parallelised back in 2006, there has been a certain randomness in how fragments and multi-block files are ordered in the output filesystem even if the input remains the same. This is because the multiple parallel threads can be scheduled differently between Mksquashfs runs. For example, the thread given fragment 10 to compress may finish before the thread given fragment 9 to compress on one run (writing fragment 10 to the output filesystem before fragment 9), but, on the next run it could be vice-versa. There are many different scheduling scenarios here, all of which can have a knock on effect causing different scheduling and ordering later in the filesystem too. Mkquashfs doesn't care about the ordering of fragments and multi-block files within the filesystem, as this does not affect the correctness of the filesystem. In fact not caring about the ordering, as it doesn't matter, allows Mksquashfs to run as fast as possible, maximising CPU and I/O performance. But, in the last couple of years, Squashfs has become used in scenarios (cloud etc) where this randomness is causing problems. Specifically this appears to be where downloaders, installers etc. try to work out the differences between Squashfs filesystem updates to minimise the amount of data that needs to transferred to update an image. Additionally, in the last couple of years has arisen the notion of reproducible builds, that is the same source and build environment etc should be able to (re-)generate identical output. This is usually for verification and security, allowing binaries/distributions to be checked for malicious activity. See https://reproducible-builds.org/ for more information. Mksquashfs now generates reproducible images by default. Images generated by Mksquashfs will be ordered identically to previous runs if the same input has been supplied, and the same options used. 1.1.1 Dealing with timestamps Timestamps embedded in the filesystem will stiil cause differences. Each new run of Mksquashfs will produce a different mkfs (make filesystem) timestamp in the super-block. Moreover if any file timestamps have changed (even if the content hasn't), this will produce a difference. To prevent timestamps from producing differences, the following new Mksquashfs options have been added. 1.1.2 -mkfs-time