Train on gpu predict on cpu?

[xinqinglxl]

Hello, everyone. I'm new to Caffe2 and now working on my toy demo. I wonder whether I can train my model with gpu and use(predict) it on cpu. Any advice?

[orionr]

Hi @xinqinglxl. Absolutely. In fact, that's very common. Typically the model is trained on GPU (with an EC2 server, for example) and then can be run, at inference / predict time, on ARM CPU for mobile or Intel for server-side. Hope that helps. Take a look at the tutorials for more details on the mobile case:

https://caffe2.ai/docs/tutorials.html

[xinqinglxl]

Hi, @orionr. thanks for the advice, it's do a lot help. But it seems the deploy model is loaded by C, I wonder if I can do the same thing with Python.

[danigamba]

Hi @xinqinglxl, i can confirm that it works flawlessly with python too. We are training models using caffe2 on CUDA machines and then execute the trained networks on both x86 and ARM processors without gpu support, you will just get a warning on import.

[orionr]

@xinqinglxl you can load it at runtime via C++ or Python. If you want to run it on mobile you'd use C++ to load the model since Python isn't typically available. On server CPU, though, by all means you can use Python as @danigamba mentioned.

[xinqinglxl]

@danigamba @orionr thanks for the reply. I tried to load the GPU-trained model and run it with cpu, but at the moment I got the following error, anything wrong?

RuntimeError: [enforce fail at blob.h:94] IsType(). wrong type for the Blob instance. Blob contains caffe2::Tensor while caller expects caffe2::Tensor .

[rribani]

Same error here.

[suryafyi]

Did someone find a solution to this? Same here