search

triton-inference-server / server

The Triton Inference Server provides an optimized cloud and edge inferencing solution.
https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html
BSD 3-Clause "New" or "Revised" License
8.16k stars 1.46k forks source link

Failed to load model - Unknown Builtin Op Torch Sparse #4101

Closed mprithiv closed 2 years ago

mprithiv commented 2 years ago

Description Failed to load a GNN model failed to load 'gat' version 1: Internal: failed to load model 'gat': Unknown builtin op: torch_sparse::ptr2ind. Could not find any similar ops to torch_sparse::ptr2ind. This op may not exist or may not be currently supported in TorchScript. Triton Information What version of Triton are you using? 22.02-py3 Are you using the Triton container or did you build it yourself? Triton To Reproduce Steps to reproduce the behavior.

Describe the models (framework, inputs, outputs), ideally include the model configuration file (if using an ensemble include the model configuration file for that as well). GATConv, PyG Expected behavior A clear and concise description of what you expected to happen.

CoderHam commented 2 years ago

@mprithiv torch_sparse is not part of the vanilla pytorch library is it? Triton does not include the miscellaneous extension libraries for pytorch.

mprithiv commented 2 years ago

No it is not a part of vanilla pytorch but it is a required package for PyG model I am using. Is there any way to include packages like torch_scatter, torch_sparse with the deployment?

deadeyegoodwin commented 2 years ago

Here is documentation on adding custom pytorch operations: https://github.com/triton-inference-server/server/blob/main/docs/custom_operations.md#pytorch