search

steveseguin / raspberry_ninja

Publish or capture VDO.Ninja streams with Python (Raspberry Pi, Linux, Mac, Windows WSL)
https://raspberry.ninja
144 stars 28 forks source link

Raspberry Pi image compression - Deflate64 incompatible w Linux #24

Closed gerbrent closed 2 years ago

gerbrent commented 2 years ago

Diving into the project for the first time and excitedly so!

However, the first roadblock: The Raspberry Pi image provided is compressed via Deflate64, presumably since the image was compressed with "Microsoft Windows Explorer which uses Deflate64 compression when creating ZIP files larger than 2GB" [1]

This becomes problematic on Linux - at the very least on Fedora 36 - since "support for Deflate64 in the open-source ecosystem is very poor. Most ZIP libraries have declined to implement Deflate64, citing its proprietary nature." [1]

[1] https://pypi.org/project/zipfile-deflate64/

A workaround

p7zip, the POSIX-compatible CLI version of 7zip, includes Deflate64 compatibility. This was available in the Fedora repositories to install via sudo dnf install p7zip. Not available on stock Ubuntu... see more details at [1]

Proposed Solution

I would like to propose compressing the image with a more widely compatible compression option. Even the documentation here states:

(** note, Someone reported that their macOS system could not unzip it, but worked fine on Windows)

This feels like an unnecessary hurdle for newcomers to the project who originate from platforms other than Windows, and could easily discourage adoption.

steveseguin commented 2 years ago

Thank you for reporting and identifying the issue:

For the time being, I've updated the download with a 7zip version of the zip. I've tested it with macOS and will test it with Ubuntu:

https://drive.google.com/file/d/1JcQFWNRGzQNMbL9JP8VIRHj0EhTMNqpr/view?usp=sharing

I can host the file as just a *.img file as well if Ubuntu as issues, or as gzip.

steveseguin commented 2 years ago

It worked on a pretty close to stock Ubuntu 18.04 as well. Funny thing is, I had 7zip already installed, and often use it; just didn't think it would have offered much of a difference in this case.

Anyways, if you think a *.img uncompression version would be helpful, I can do that too, but I have limited cloud drive space is all.

Kindly, Steve

gerbrent commented 2 years ago

wow @steveseguin - thanks for being so responsive!

I just downloaded the new file from your link above and it works wonderfully from Fedora KDE/Dolphin file browser w standard workflow (right click, extract...)

Uncompressed .img: I feel there's really no need for this option, especially at about twice the size for little gain in functionality now that you've addressed the above..

Thank you!