search

unitaryai / detoxify

Trained models & code to predict toxic comments on all 3 Jigsaw Toxic Comment Challenges. Built using ⚡ Pytorch Lightning and 🤗 Transformers. For access to our API, please email us at contact@unitary.ai.
https://www.unitary.ai/
Apache License 2.0
927 stars 114 forks source link

Running this on mobile with pytorch mobile #60

Open lee-hodg opened 2 years ago

lee-hodg commented 2 years ago

Is there anyway I can load the weights and "quantize" to reduce the size so I can run with pytorch mobile client side?

laurahanu commented 2 years ago

Hello,

Currently all our models are trained with full precision so reducing the precision only at inference time could be unsafe. However, it should be possible and the torch docs on quantization should be helpful. We are planning to look more into a mobile version of Detoxify in the future, and of course, contributions are welcome!