imxcstar / RWKVSharp

call rwkv v4/v5/v6 raven/world/finch 1B5-14B ggml using csharp cpu/gpu (support INT4,8,Float16,32)
MIT License
31 stars 4 forks source link

Does ONNX models used in this project supports CUDA only( crashes on CPU) #1

Open JohnMasen opened 1 year ago

JohnMasen commented 1 year ago

HI, I'm trying to run your models with CPU, the application crashes while loading the model. To verify if these ONNX models could be loaded by CPU, I created a simple test app like this.

using Microsoft.ML.OnnxRuntime;
using Onnx;
SessionOptions options = new();
options.AppendExecutionProvider_CPU();
InferenceSession session = new InferenceSession(@"D:\Models\RWKV-4-Raven-3B-v11-Eng49-Chn49-Jpn1-Other1-20230429-ctx4096_32_2560_16.onnx", options);
Console.WriteLine("Done");
Console.ReadLine();

the application crashes with following exception

System.AccessViolationException: 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'

Could you please kindly suggest if any thing I did wrong or the model bin file is designed for CUDA only?

imxcstar commented 1 year ago

Which onnx nuget package are you using? The CPU does not need to install cuda when using this'Microsoft.ML.OnnxRuntime '

JohnMasen commented 1 year ago

Microsoft.ML.OnnxRuntime v1.14.1

imxcstar commented 1 year ago

This is not clear. Perhaps the configuration environment is different somewhere. 😂