Sparsify custom pytorch models from scratch

[anberto]

Hi, I have a model that cannot be traced back to any of the default supported architectures (yolo, llm, trasformers...). I would like to see the benefits of sparsification on it.

The only integration example I have found that sparsify from scratch is the one using yolo5 (link).

It would be great to have an integration example tackling a custom local pytorch model.

I hope I have not missed anything that might have already been written on the topic. In that case, I'd be happy to be proven wrong.

Thanks all

[yoloyash]

Hey, did you find a way to do it?

[anberto]

Hi, not exactly. Starting from the integration tutorial https://github.com/neuralmagic/sparseml/blob/main/integrations/torchvision/tutorials/docs-torchvision-sparsify-from-scratch-resnet50-beans.ipynb and with the help of a skilled colleague I understood it is just a matter of pytorch programming knowledge. One needs to define its data loader to feed its model and train it. But the train loop is no different from the usual pytorch pattern.

I guess the real challenge is to write down effective recipes. Here is where I stumbling right know. I am not able find resources about available recipe operators and suggestions on how to compose them. I guess the best way is to download them from sparsezoo and study them.

[yoloyash]

@anberto Thanks! I did the same. I am trying to sparsify YOLOv10. I understand how to sparsify it using pytorch and sparseml, I just don't understand what layers to pick and how much sparsity each one should have. I am studying the YOLOv8 and YOLOv5 tutorials but they haven't really given an explanation for the layers, I guess it is decided empirically/experimentally.

[anberto]

@yoloyash I jumped to the same conclusions. From here https://github.com/neuralmagic/sparsezoo?tab=readme-ov-file#model-directory-and-file and here https://github.com/neuralmagic/sparsezoo?tab=readme-ov-file#model-directory-and-file it looks like it is possible to get the recipes of a sparsezoo model via python. Good luck and happy study

[bfineran]

For a basic integration, take a look at the top level readme - the key point to get started is making sure that the recipe can wrap the model, dataset, and optimizer. Since we wrap the optimizer step function, once this is done the recipes should be applied

[anberto]

Hi @bfineran Is there a place that documents how to write recipes? The operators that can be used and their parameter, syntax ecc? While surfing the internet I can only find broken links, like https://docs.neuralmagic.com/sparseml/source/recipes.html. Thanks

[bfineran]

Hi @anberto we're currently in the middle of a large architecture change and will be updating our docs with it. In the meantime, looking at recipes in our examples may be the best bet. Searching for the class implementation of each modifier and looking at the types in the init function as well as their docstrings will help understand the valid parameters

[jeanniefinks]

@anberto As there have been no further comments, I will go ahead and close out this issue. Thank you for reaching out! If you have not already, be sure to "star" our GitHub repos.

Best, Jeannie / Neural Magic