search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8.21k stars 1.47k forks source link

Windows Tensorflow and Pytorch support #3715

Closed umutozturk95 closed 2 years ago

umutozturk95 commented 2 years ago

Hi, I am trying to compile triton server in Windows 10 Pro (Docker support) platform . But As far as I can see from the build process there is no tensorflow and pytorch support. I created C++ application using vs 2015 . I want to produce a model result from c++ app using tensorflow or pythorc over triton server. I want to use triton server to switch between these deep learning libraries easily.So the triton server should be in the local system, not in the cloud environment. That's why I'm thinking of embedding the triton server into the software using C api. Do you think this process can be done? Do i need to pay attention? Thanks.

umutozturk95 commented 2 years ago

I also get an error when compiling buil.py with Dockerfile.win10.min. win10-py3-min image is created by using docker -t command successfully. I am sending the error tracs in the error_list.txt file. How can i solve the problem ? Thanks.

error_list.txt .

CoderHam commented 2 years ago

@umutozturk95 if you are building a specific release branch you need to use the appropriate repo branch tags for that release. You should use r21.xx for core, common, backend, client and tf/pytorch and other respective repos if you are using the r21.xx branch of the server repo

umutozturk95 commented 2 years ago

@CoderHam @dzier Thanks. I solved this build problem due to branch conflict. When I successfully completed the build process, I saw that there were no tensorflow and pythorch folders for the backend in Windows 10 platform.But there are only build files of onnx ,openvivo and tensorrt.Does triton provide tensorflow and pytorch support on windows platform? I couldn't understand how to use tensorflow /pytorch support on windows Note: I did the build process you described for windows in build tutorial page.I get error when compiling build.py due to libtriton_tensorflow.so and libtriton_pytorch.so files.

dzier commented 2 years ago

At this time, windows support is an alpha release and very limited. It does not currently support Tensorflow or Pytorch. The windows release only currently supports ONNX RT, OpenVino, and TensorRT. You can try to use something like Model Navigator to convert your Tensorflow or Pytorch model to TensorRT and then run it on Windows.

Closing issue for now, please re-open if you have more questions or issues.