conda / conda-pack

Package conda environments for redistribution
https://conda.github.io/conda-pack/
BSD 3-Clause "New" or "Revised" License
508 stars 91 forks source link

Add support for alternative compressors for mksquashfs #274

Closed dmargala closed 1 month ago

dmargala commented 1 year ago

Checklist

What is the idea?

Add a conda-pack cli option to specify a compressor for mksquashfs or have conda-pack automatically select a compressor that is supported by the mksquashfs in the user's environment.

Why is this needed?

Currently, conda-pack does not allow the user to select an arbitrary compressor for mksquashfs. It looks like there is a heuristic that uses the --compress-level cli option to select a compressor but the only compressors it can choose from are zstd or xz. The mksquashfs in my environment does not support either of those, it looks like my mksquashfs only supports gzip and lz4. Adding support for alternative compressors would allow users with mksquashfs that do not support zstd or xz to use compression when packing environments to squashfs.

What should happen?

From a user perspective, I would like to see a conda-pack cli option to specify a compressor or have conda-pack automatically select a compressor that is supported by the mksquashfs in my environment if possible.

Additional Context

It looks like there are conda-pack squashfs compression performance tests documented in #176 and there has been discussion about which compressors to support in #177. I appreciate the log message solution arrived at there, it helped me figure out where to look next. However, the suggestion to use xz instead of zstd is not always viable.

github-actions[bot] commented 1 month ago

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

  1. Verify that you can still reproduce the issue at hand
  2. Comment that the issue is still reproducible and include:
    • What OS and version you reproduced the issue on
    • What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!

dmargala commented 1 month ago

It turns out that the version of mksquashfs that is available on the system I am using is a custom static build with limited compression support. I don't have any control over that but I can work around the issue by installing the squashfs-tools package in my conda environment.