[ ] Changes to dev-tools e.g. CI config / github tooling
[ ] Docs
[x] Bug fixes
[ ] Code refactor
What is the current behavior? (You can also link to an open issue here)
I have a multi-GPU setup and wanted to distribute a model across multiple devices (using -M device=auto via the CLI).
HuggingFaceAPI currently uses self.device for both the model's device_map arg as well as the input's device. This resulted in trying to send inputs to auto, causing an error.
What is the new behavior?
Now inputs are sent to the (first) device associated with the model
Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
No, this is backwards-compatible
Other information:
I didn't open an issue since the fix was so quick, but I'm happy to do that or follow other conventions for the project.
This PR contains:
What is the current behavior? (You can also link to an open issue here)
I have a multi-GPU setup and wanted to distribute a model across multiple devices (using
-M device=autovia the CLI).HuggingFaceAPIcurrently usesself.devicefor both the model'sdevice_maparg as well as the input'sdevice. This resulted in trying to send inputs toauto, causing an error.What is the new behavior?
Now inputs are sent to the (first) device associated with the model
Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
No, this is backwards-compatible
Other information:
I didn't open an issue since the fix was so quick, but I'm happy to do that or follow other conventions for the project.