search

NVIDIA / nvcomp

Repository for nvCOMP docs and examples. nvCOMP is a library for fast lossless compression/decompression on the GPU that can be downloaded from https://developer.nvidia.com/nvcomp.
Other
559 stars 78 forks source link

[FEA] Configure compression level/speed #81

Open technillogue opened 1 year ago

technillogue commented 1 year ago

Is your feature request related to a problem? Please describe. I'm using nvCOMP to load data from disk to GPU, and don't care about compression but do care about getting the best compression ratio possible.

Describe the solution you'd like CompressionConfig has settings for each algorithm, for example gdeflate could max_hash_checks and lazy vs greedy matching.

Describe alternatives you've considered If this is available through the low-level API, better documentation would be helpful.

eschmidt-nvidia commented 1 year ago

Hi technillogue,

In the case of GDeflate we do have a "high compression" mode. This is documented in include/nvcomp/gdeflate.h, and was designed for such use cases as yours where compression speed is less of a concern.

We're also considering adding similar modes to zSTD in an upcoming release.

-Eric

github-actions[bot] commented 1 year ago

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

technillogue commented 1 year ago

Setting anything but 0 for Gdeflate gives:

Invalid format_opts.algo value (high compression option (1) not currently supported) Invalid format_opts.algo value (entropy only option (2) not currently supported)

github-actions[bot] commented 1 year ago

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

eschmidt-nvidia commented 1 year ago

We changed the HLIF for 3.0 to call the LLIF. Now the other algorithms should work for the GDeflate HLIF.

github-actions[bot] commented 11 months ago

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.