strange error when using cuda: 0xc0000409, STATUS_STACK_BUFFER_OVERRUN

[altunenes]

I'm encountering an issue when trying to use the CUDA provider with sherpa-rs on Windows.

I use the latest versions of CUDA, ONNX, and other dependencies. On my machine, CUDA works fine with my other Rust ONNX Runtime projects So I am sure it is installed and working correctly.

When I try to utilize the models with cuda, I get the following error, with CPU, no problem at all. Even when I type coreml, it just says "CoreML is for Apple only. Fallback to cpu!" and it keeps working.

let mut recognizer: WhisperRecognizer = WhisperRecognizer::new(
            "tiny-decoder.onnx".into(),
            "tiny-encoder.onnx".into(),
            "tiny-tokens.txt".into(),
            "sv".into(),  
            Some(false),
            Some("cuda".to_string()),
            None, 
            None, 
        );

sherpa-rs = { version = "0.1.7", features = ["cuda"] }

Using CUDA provider fatal runtime error: Rust cannot catch foreign exceptions error: process didn't exit successfully:target\debug\test_vad.exe(exit code: 0xc0000409, STATUS_STACK_BUFFER_OVERRUN)

I also did cargo clean, and cargo run again but its the same. I found this discussion but its not clear to detect the source error: https://users.rust-lang.org/t/status-stack-buffer-overrun/34873/4

[thewh1teagle]

Thanks for report. I suspect that it's related to linking issue / missing DLLs. I released new version. Can you try again with the new version?

sherpa-rs = "=0.1.8-beta.0"

Also pass Some(true) to WhisperRecognizer for debug

[altunenes]

unfortunately its the same. sherpa-rs = { version = "=0.1.8-beta.0", features = ["cuda"] }

Note that the CPU version with this feature flag does not work either:

    let extractor_config = speaker_id::ExtractorConfig::new(
        "3dspeaker_speech_eres2net_sv_en_voxceleb_16k.onnx".into(),
        None, 
        None,
        true,
    );

but if I use Some("coreml".to_string()), at least it works on cpu."sion.cc:GetSessionOptionsImpl:198 CoreML is for Apple only. Fallback to cpu!"

To able to run in CPU properly I should simply use sherpa-rs = { version = "=0.1.8-beta.0"] } (without cuda flag)

I don't know, maybe it's my machine. Window Updates might have broken something. But I would like to say that my other onnx projects are working correctly with Cuda (rust & python).

Debug:

est_vad\target\debug\build\sherpa-rs-sys-398016b229ca6795\out\sherpa-onnx\sherpa-onnx\csrc\speaker-embedding-extractor-impl.cc:GetModelType:38 language=English
framework=3d-speaker
feature_normalize_type=global-mean
comment=This model is from iic/speech_eres2net_sv_en_voxceleb_16k
url=https://www.modelscope.cn/models/iic/speech_eres2net_sv_en_voxceleb_16k/summary
sample_rate=16000
output_dim=192
normalize_samples=1

fatal runtime error: Rust cannot catch foreign exceptions
error: process didn't exit successfully: `target\debug\test_vad.exe` (exit code: 0xc0000409, STATUS_STACK_BUFFER_OVERRUN)

[thewh1teagle]

Looks like there's upstream issue with sherpa-onnx. I crash for me too with the following:

build commands

*Build it with:* **Note**: Add `-DSHERPA_ONNX_ENABLE_GPU=ON` to enable cuda. ```console git clone https://github.com/k2-fsa/sherpa-onnx cd sherpa-onnx cmake -B build . -DCMAKE_BUILD_TYPE=Release -DSHERPA_ONNX_ENABLE_BINARY=ON cmake --build build -j8 ``` *Prepare Whisper* ```console winget install -e --id JernejSimoncic.Wget winget install -e --id GnuWin32.Tar wget.exe https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-whisper-tiny.tar.bz2 tar.exe xvf sherpa-onnx-whisper-tiny.tar.bz2 ``` *Run with* ```console .\build\bin\Debug\whisper-c-api.exe ``` If it works, try to change the provider from `cpu` to `cuda` in the file `whisper-c-api.c`: ```console cmake --build build -j8 ``` Then run again.

Related: https://github.com/k2-fsa/sherpa-onnx/issues/1217

[csukuangfj]

By the way, I think you should add CI tests to this project.

The current CI only ensures the project can be built without issues. It would be great if you also test that it runs without any problems.

[thewh1teagle]

@altunenes I missed build flag in the previous comments and that cause the issue. Please try to build it with the following commands (Don't forget to include the flag to enable cuda)

build commands

*Build it with:* **Note**: Add `-DSHERPA_ONNX_ENABLE_GPU=ON` to enable cuda. ```console git clone https://github.com/k2-fsa/sherpa-onnx cd sherpa-onnx cmake -B build . -DCMAKE_BUILD_TYPE=Release -DSHERPA_ONNX_ENABLE_BINARY=ON cmake --build build --config Release -j8 ``` *Prepare Whisper* ```console winget install -e --id JernejSimoncic.Wget winget install -e --id GnuWin32.Tar wget.exe https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-whisper-tiny.tar.bz2 tar.exe xvf sherpa-onnx-whisper-tiny.tar.bz2 ``` *Run with* ```console .\build\bin\Debug\whisper-c-api.exe ``` If it works, try to change the provider from `cpu` to `cuda` in the file `whisper-c-api.c`: ```console cmake --build build --config Release -j8 ``` Then run again.

[thewh1teagle]

By the way, I think you should add CI tests to this project.

The current CI only ensures the project can be built without issues. It would be great if you also test that it runs without any problems.

I added CI tests. Now we can see clearly the issue reported here.

https://github.com/thewh1teagle/sherpa-rs/actions/runs/10242004087/job/28331095556

DirectML works (shared lib)

By the way there's another issue with cuda on ubuntu https://github.com/thewh1teagle/sherpa-rs/actions/runs/10242004087/job/28331095754

And I checked cuda version. I use the correct one (12.x)

[csukuangfj]

》 Now we can see clearly the issue reported here.

There are NO GPUs on the CI machines.

Are you able to reproduce locally on your Windows NVIDIA GPU machine?

Please also try cuda 11.8.

(It was cuda 11.8 in the onnxruntime doc. It is somehow changed to 12.x someday for some unknown reason.)

[altunenes]

》 Now we can see clearly the issue reported here.

There are NO GPUs on the CI machines.

Are you able to reproduce locally on your Windows NVIDIA GPU machine?

Please also try cuda 11.8.

(It was cuda 11.8 in the onnxruntime doc. It is somehow changed to 12.x someday for some unknown reason.)

just to make sure, sherpa-onnx supports Cuda 12.x, with the latest Onnx right? My windows machine really slow nowadays and my tests with different envs settings takes really time, sorry for the delays :(

[csukuangfj]

We only support onnxruntime 1.17.1

[altunenes]

Oh, I missed that important detail! I'm so sorry! :-( so this is the source of the error then. Thank you so much. I reproduced this error on the latest CUDA and ort.

I close the issue for now (feel free to open it again if you have a comment on any other detail).

[csukuangfj]

does it work for onnxruntime 1.17.1?

[altunenes]

does it work for onnxruntime 1.17.1?

As I said, for now it is not possible to test this on my current work machine. My current work machine is really slow on these days and it is quite difficult for me to switch to a different environment. But hopefully, I will test this soon.