gzipped firmware .bit files?

[smoe]

Is your feature request related to a problem? Please describe. The disk estate occupied by the firmware is impressive, i.e. >>100 MB. Most of that disk space is occupied by .bit files, which is mostly redundant, e.g.:

 od -c mesact/src/firmware/5i25/5i25_7i85s_7i78.bit | tail
1231120  \0  \0  \0  \0 200 001  \0  \0  \0 021  \0 001 200 001  \0  \0
1231140  \0 021  \0 001  \0  \0  \0 032 233  \f      \0      \0      \0
1231160      \0      \0      \0      \0      \0      \0      \0      \0
*
1231220      \0      \0      \0      \0      \0   0 241  \0  \n   0 241
1231240  \0 003      \0      \0      \0      \0   0 241  \0  \n   0 241
1231260  \0 005   0 341  \0 377   0 301  \0 201   0 002  \0   / 337 322
1231300   0 241  \0  \r      \0      \0      \0      \0      \0      \0
1231320      \0      \0      \0      \0      \0      \0      \0      \0

Describe the solution you'd like Even for small .bit files, the compression achievable is considerable:

$ du -sh mesact/src/firmware/5i25/5i25_7i85s_7i78.bit
336K    mesact/src/firmware/5i25/5i25_7i85s_7i78.bit
$ gzip mesact/src/firmware/5i25/5i25_7i85s_7i78.bit
$ du -sh mesact/src/firmware/5i25/5i25_7i85s_7i78.bit*
96K     mesact/src/firmware/5i25/5i25_7i85s_7i78.bit.gz

and for larger .bit files I observed a compression to ~15%.

Describe alternatives you've considered

Please consider to redistribute the .bit files after compression and teach mesact to read those compressed files directly.

[jethornton]

The thought just came to me to use a plugin method for each board. Maybe distribute with only the currently available boards firmware and for other boards have the use download and install a plugin for that board... thinking out loud so to speak. The .bit files are machine code files and not text files so for Mesaflash to use them I assume they need to be not compressed. The firmware directory is 200 MB, libmesact is 1.6 MB and mesact and ui is 1.2 MB, manuals is 10 MB so looks like a huge size reduction to not include all firmware and pin files with mesact.

I guess off the top of my head each plugin would need a repo to store the files... gotta think on that a bit.

JT

[smoe]

Thank you for your kind and constructive reply. I think as a first shot it is just fine to ship the files compressed, maybe even with xz, which is more efficient than gzip and should get the firmware directory <50MB, which sounds like good enough as a start. Then, instead of downloading from somewhere, the user (or mesact) needs to run xz -d prior to executing mesaflash - that is done so quickly for those 304k files, one barely notices it - just the 11MB one may be noted on older machines. Further, it saves you considerable dev time and (more importantly) we can just flash it to the LinuxCNC USB stick so this can be used on one of the many CNC machines without Internet access.

I would actually use /tmp for the uncompressed file and not touch anything that you installed to /usr, as in xzcat firmware.bit.xz > /tmp/firmware.bit and then run mesaflash on /tmp/firmware.bit.

But what this really means to me is that sooner or later we need to teach mesaflash some new tricks. I do not see why mesaflash cannot just work with the compressed file (and uncompress internally). That should then even be faster on an old machine than reading the larger file from disk :)

[jethornton]

I decided it was an easier and cleaner way if I moved the firmware files to another repository and have the user download only firmware for the board they are using. When they download a firmware package for a board it's installed in ~/.local/lib/libmesact/name of board.

JT

[smoe]

@jethornton May I carefully ask if you could find a similar solution for the PDFs? Those are also distributed without their sources being available in the distribution and would thus violate Debian's Free Software Guidelines.