search

microsoft / hummingbird

Hummingbird compiles trained ML models into tensor computation for faster inference.
MIT License
3.32k stars 274 forks source link

if i have saved a pytorch_based model, i want to run inference on cpu, how to change the codes? #739

Closed Bobby-youngking closed 8 months ago

Bobby-youngking commented 8 months ago

from sklearn import datasets from sklearn.ensemble import RandomForestClassifier from hummingbird.ml import convert, load iris = datasets.load_iris() X, y = iris.data, iris.target rfmodel = load('hb_model') rfmodel.to('cpu') print(rfmodel.predict(X))

i tried this way, but the result showed that i was running on gpu

interesaaat commented 8 months ago

Hi! Just change rfmodel.to('cpu') to rfmodel = rfmodel.to('cpu'). This should work.

mshr-h commented 8 months ago

I think we have to update the readme.

https://github.com/microsoft/hummingbird#examples

# Use Hummingbird to convert the model to PyTorch
model = convert(skl_model, 'pytorch')

# Run predictions on CPU
model.predict(X)

# Run predictions on GPU
model.to('cuda')
model.predict(X)
ksaur commented 8 months ago

Was this an API change in a newer version of torch? Because the README example definitely works, or used to!

ksaur commented 8 months ago

I was not able to reproduce this issue with the latest (torch==2.1.0), and the example in the documentation still worked as written for me.

@Bobby-youngking can you please post which versions you are using and a bit more info? you shouldn't have to reassign the model

ksaur commented 8 months ago

Closing due to inactivity and we cannot repro, please reopen if you still have problems with this.