search

NVIDIA / nvcomp

Repository for nvCOMP docs and examples. nvCOMP is a library for fast lossless compression/decompression on the GPU that can be downloaded from https://developer.nvidia.com/nvcomp.
Other
557 stars 79 forks source link

[QST] Selecting Cascade parameters #68

Open Shamazo opened 2 years ago

Shamazo commented 2 years ago

This blog post from December 2020 has these sentences:

 To get the best performance using cascaded compression, it is helpful to select the best configuration for each input dataset. To accomplish this, we are working on an auto-selector that analyzes a given dataset and quickly determines the best cascaded compression configuration to use. This auto-selector will be available with a new version of nvcomp in an upcoming release

Was this auto-selector ever released? and if not, how can I determine the best Cascaded compression configuration for my dataset?

Thanks

github-actions[bot] commented 2 years ago

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

ndickson-nvidia commented 1 year ago

There was a cascaded selector implementation in version 2.0 and 2.1, but when the big API change was done for 2.2, it didn't get ported over. It's come up a few times, but it's usually the case that either a good guess is known a priori what sort of configuration would be useful for a particular use case, or it's feasible to find a representative data set to compare results on with just a shell script just running benchmark programs. The difficulty is that the overhead from dynamically trying to determine the configuration to use is not worth it for those cases, and in other cases, often other compressors would be faster/better anyway, taking into account the overhead.

If you have a use case where cascaded compression would be useful and dynamic selection of the best configuration at compression time could be worthwhile, please let us know! Thanks.

ndickson-nvidia commented 1 year ago

Oh, and if you'd like tips on statically finding a good cascaded configuration for a particular use case, feel free to ask and provide a bit of info on the use case if possible, because I can give some tips.

github-actions[bot] commented 1 year ago

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

github-actions[bot] commented 1 year ago

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.