search

torch / torch7

http://torch.ch
Other
9k stars 2.38k forks source link

Feature request - documentation/guideline for deployment of torch trained network into a C/C++ application #896

Open ghost opened 7 years ago

ghost commented 7 years ago

Hi All:

I am using torch for a robotics research project, and need to use the trained network in a C++ program which will be subsequently linked with ROS. However it appears that a good guideline on this topic is pretty limited.

I note the following resources: (1) Some project specifically targets at this topic, like: https://github.com/tuotuoxp/cpp-torch, but the types of network layer supported are not a lot. (2) https://github.com/torch/nn/tree/master/lib/THNN seems to be designed for such a purpose. But I do not understand the complete procedure, e.g. how to load the trained network from saved files in C++ and how to use it with THNN API. (3) Some people also recommend to use inter-process communication. Is there any example/guide on using IPC between a torch deep network server and a C++ client app, and how to do a two-way communication?

I believe this feature is really important and it should become an official part of torch framework. Thanks a lot for the help!

aaalgo commented 7 years ago

I've done some research and made it work in one of my private projects, and have just re-wrote that piece of code here: https://github.com/aaalgo/xnn/blob/master/torch.cpp . (XNN is my C++ wrapper that supports caffe, mxnet, python and now torch, with a unified API.) I haven't have time to tested the code yet, as I just added that in response to your ticket, but it should give you an idea of how it is done. Lua is designed to be embedded into a C/C++ program. I think the actual solution is relatively clean and works very well. I'm able to serve the OpenFace model with a multi-threaded server, each thread running it's own Lua instance single-threaded. I hope you find this useful while waiting for an official solution.

OpenFace python API actually takes the sub-process approach, with communication done by piping stdin/stdout. I found direct integration with C++ much easier and more efficient.

ghost commented 7 years ago

@aaalgo Thanks very much for your generous help. The example looks pretty clear and will try it later. I Believe such clear examples should be in official documentation as well.