
inicomp is a depacker stage for lDOS's (and RxDOS's) iniload kernel.
It currently supports six compression formats, namely: BriefLZ, LZ4,
Snappy, exomizer, X compressor, and heatshrink.

For usage conditions, refer to comments in the source files.


Example usage (without _IMAGE_EXE, with _BRIEFLZ):

inicomp$ blzpack /path/to/rxdos-7.2x/ldos/lDOSLOAD.BIN lDOSLOAD.BLZ
inicomp$ ./mak.sh -I/path/to/lmacros/
rxdos-7.2x/ldos$ nasm -Ox -D_REVISIONID="'hg $(hg -R .. id -i)'" \
 -I ../lmacros/ -f bin -o ../bin/RxDOS.COM -l ../lst/RxDOS.LST \
 -D_MAP=../map/RxDOS.MAP ../ldosboot/iniload.asm \
 -D_PAYLOAD_FILE=/path/to/inicomp/iniblz.bin


Example usage with _IMAGE_EXE (with _BRIEFLZ) payload:

ldebug/source$ use_build_revision_id=1 ./make -D_BOOTLDR -D_DEBUG4 -D_DEBUG2
inicomp$ blzpack /path/to/ldebug/bin/debug.com lDOSLOAD.BLZ
inicomp$ ./mak.sh -I/path/to/lmacros/ -D_IMAGE_EXE -D_EXEC_OFFSET=32
ldosboot$ nasm -D_MAP=iniload.map -l iniload.lst \
 -I../lmacros/ -o LDEBUGPK.COM \
 -D_PAYLOAD_FILE="'/path/to/inicomp/iniblz.bin'" \
 -D_IMAGE_EXE_MIN_CALC="((($(stat -c %s /path/to/ldebug/bin/debug.com) + \
  _IMAGE_EXE_AUTO_STACK) + 15) & ~15)" \
 -D_IMAGE_EXE_AUTO_STACK=2048 -D_IMAGE_EXE=1 -D_EXEC_OFFSET=32 iniload.asm


Currently, the system must have enough memory to hold all of the following:

* 20 KiB of buffers (8 KiB FAT sector, 8 KiB DMA boundary buffer) and stack
  and boot sector, at the top of the LMA (below RPL or EBDA or video memory).

* The depacker's INIT1 section, which contains the actual depacker
  and a bit of handling around it. This is directly below the "load top",
  which gives the segment of the lowest buffer at the LMA top.
  (INIT0 and INIT1 together are smaller than 2 KiB currently. As iniload's
  loader is 2 KiB, this means the uncompressed part is less than 4 KiB.)

* The compressed image.

* X compressor format needs additional memory: 256 KiB for the context
  tables, and some more to support multi-layer decompression. (16 KiB
  are currently allocated for decompressing the subsequent layers.
  Compressing the RxDOS kernel uses two layers currently.)

* The decompressed image. This is placed at linear 00600h when booting,
  or (with _IMAGE_EXE) after the PSP when started as application.

Note:	The compressed image and decompressed image may now overlap.
	 To succeed, the write pointer has to stay below-or-equal the
	 read pointer. Otherwise the decompression errors out.

The .EXE mode specifies how much memory is needed in the executable
header, so a load failure due to lack of memory can be detected by the
operating system's process loader. It can also choose a large enough
block, so that eg loading in a small UMB doesn't lead to a failure.


Possibilities:

* init0_movp can always move DN because the payload-and-INIT1 destination is
  always above-or-equal the source of that move. (Reverted to optimise to an
  UP movement if the source and destination do not overlap.)

* The depacker could support overlapping source and destination buffers,
  in which case it should check that the destination write pointer stays
  below the source read pointer. As the kernel is typically below 128 KiB
  in size, this is only needed for systems with 256 KiB or less of LMA.
  (done: _ALLOW_OVERLAPPING)

* _IMAGE_EXE could be supported, with some adjustments to the segmentation
  to allow the same image to be entered either as kernel or as MZ executable.
  (done: _IMAGE_EXE, though sloppily)

* A device driver entry to the .EXE image could be supported. Neither the
  RxDOS kernel nor the lDebug debugger have such an entrypoint yet.

* Optimisations are possible if the block size is decreased, however,
  I prefer it to support any block size, which it does for now.

* CRC checks of compressed or decompressed data could be added.

* If a kernel CONFIG section is added to iniload or lDOS init, inicomp
  should also support this. (Embed one in the uncompressed header, pass it on
  to the kernel when loading.)

* Handling and structures could be added to allow a to-be-written utility
  to compress and decompress images within existing kernel files. However,
  typically the compressing during building the kernel is sufficient.
  (The FreeDOS kernel and EDR-DOS's also do not have such utilities.)

* Other compression algorithms could be used. (several done, six in total)

* If the depacker stage was integrated into iniload, some common code could
  be shared, mainly the error handling.


Comparison of the formats:

The example is building the RxDOS kernel revision d081d3c5c97e with the
following command line parameters:

build_inicomp_options=-D_DEBUG0\ -D_COUNTER=64 use_build_decomp_test=1 ./mak.sh

Compr format	File size
none		101376
brieflz		 56320
lz4		 55296
snappy		 60416
exodecr		 47616
x		 68608
heatshrink	 57856

The exomizer format compresses best. As the depackers all need about
1 to 2 KiB, the superior compression of the payload that exomizer
manages wins.

As mentioned, the X compressor format requires a lot of memory.

X compressor and heatshrink seem to be the slowest decompressors. (The
exomizer *compressor* is slow, but that is a good tradeoff for us.)

The BriefLZ, exomizer, and X compressor depackers are ports of their
original decompressor sources. Thus, they carry the copyrights and
usage conditions of their sources. (All of these are permissive.)
The LZ4, Snappy, and heatshrink decompressors were written based on
the format descriptions or abstract program logic without directly
porting the code. Thus, they are provided under the Fair License.
