Open MithrilMan opened 3 weeks ago
The Python code creates the ONNX node proto and then runs a test on that by wrapping it in a graph proto, model proto and then finally passing it into ORT. You'd need to write out a node proto in C# into an ONNX file and then load it in. Or you can write the tokenizer out in Python and use it from C#. ML.net has examples of writing ONNX files from C#.
Hello, I'm looking for an example, in C#, about how to use SentencePieceTokenizer.
I've seen there is a py example here https://github.com/microsoft/onnxruntime-extensions/blob/main/docs/custom_ops.md#sentencepiecetokenizer
But I haven't managed to find a proper way to convert that code into a c# implementation (I'm not even sure that py code is working to be honest)
example py code is this
How can this code be translated into C#?
Beside the fetching of the model, how does the
onnx.helper.make_node
translates to?I tried with something like
but it fails with
Microsoft.ML.OnnxRuntime.OnnxRuntimeException: '[ErrorCode:InvalidArgument] No graph was found in the protobuf.'
How am I supposed to load the
SentencepieceTokenizer
in C#? How am I supposed to create that node?Thanks
P.S. If you help me sort it out, I can do a PR with working code for C# on some of the extensions