Does NNI v3.0 support multi-gpu training for model compression (pruner, distiller, quantizer)?

microsoft / nni

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

https://nni.readthedocs.io

MIT License

14k stars 1.81k forks source link

Does NNI v3.0 support multi-gpu training for model compression (pruner, distiller, quantizer)? #5687

Open DY-ATL opened 1 year ago

DY-ATL commented 1 year ago

Describe the issue: I found multi-gpu training via DataParallel is supported in v1.4 and here is the example: https://github.com/microsoft/nni/blob/v1.4/examples/model_compress/multi_gpu.py

However, I found this DataParallel related issue is still open: https://github.com/microsoft/nni/issues/3626

Can I use DataParallel for pruner or other compressors?