sony / model_optimization

Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.
https://sony.github.io/model_optimization/
Apache License 2.0
322 stars 50 forks source link

Fixed device selection #1193

Closed ambitious-octopus closed 3 weeks ago

ambitious-octopus commented 2 months ago

Pull Request Description:

This PR fixes issue #1189. This PR also allows better import of the set_working_device function, facilitating more effective DeviceManager singleton management.

Example Usage:

from model_compression_toolkit.core.pytorch import set_working_device

# Set the working device to 'GPU' or 'CPU'
set_working_device('GPU')

With this change, the set_working_device function can be easily imported and used to configure the DeviceManager's device context. This also allows a quantization to be tested on a specific device.

cc: @Idan-BenAmi

Checklist before requesting a review:

Idan-BenAmi commented 1 month ago

Hi @ambitious-octopus, we've found the root cause of this issue (model.to() error), and we think https://github.com/sony/model_optimization/pull/1204 is a better solution.