cassiebreviu / StableDiffusion

Inference Stable Diffusion with C# and ONNX Runtime
MIT License
218 stars 49 forks source link

Error load model "custom_op_cliptok.onnx" saying this is an invalid model when run the program #9

Closed pinkli closed 1 year ago

pinkli commented 1 year ago

the stack is:

Unhandled exception. Microsoft.ML.OnnxRuntime.OnnxRuntimeException: [ErrorCode:InvalidGraph] Load model from D:\Projects\StableDiffusion\StableDiffusion\text_tokenizer\custom_op_cliptok.onnx failed:This is an invalid model. In Node, ("bpetok", CLIPTokenizer, "ai.onnx.contrib", -1) : ("string_input": tensor(string),) -> ("input_ids": tensor(int64),"attention_mask": tensor(int64),) , Error Node (bpetok) has output size 2 not in range [min=3, max=3].
   at Microsoft.ML.OnnxRuntime.NativeApiStatus.VerifySuccess(IntPtr nativeStatus)
   at Microsoft.ML.OnnxRuntime.InferenceSession.Init(String modelPath, SessionOptions options, PrePackedWeightsContainer prepackedWeightsContainer)
   at Microsoft.ML.OnnxRuntime.InferenceSession..ctor(String modelPath, SessionOptions options)
   at StableDiffusion.TextProcessing.TokenizeText(String text) in D:\Projects\StableDiffusion\StableDiffusion\TextProcessing.cs:line 20
   at StableDiffusion.Program.Main(String[] args) in D:\Projects\StableDiffusion\StableDiffusion\Program.cs:line 25

on Windows 10, dotnet SDK 7.0, on branch direct-ML-EP
I built the ortextension.dll from source, as solving issue #4

pinkli commented 1 year ago

I can't find any resource regarding the "custom_op_cliptok.onnx" file. Where can I download another one, or build one, any links?

pinkli commented 1 year ago

here is why the error I guess: I built the ortextensions.dll from source, where the CLIPTokenizer model expects 3 outputs:input_ids, attention_mask and offset_mapping. But the custom_op_cliptok.onnx model in this repo, has just two outputs input_ids, attention_mask image , so the exeption has output size 2 not in range [min=3, max=3] So the solution maybe rebuilding this onnx model using the latest code, as https://github.com/microsoft/onnxruntime-extensions/blob/main/test/test_cliptok.py I'll try this another day.