CUDA 12.5 - access violation reading

[jojorne]

Compiling with CUDA 12.5 results in this error:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File ".\koboldcpp.py", line 3534, in <module>
    main(parser.parse_args(),start_server=True)
  File ".\koboldcpp.py", line 3227, in main
    loadok = load_model(modelname)
             ^^^^^^^^^^^^^^^^^^^^^
  File ".\koboldcpp.py", line 406, in load_model
    ret = handle.load_model(inputs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: exception: access violation reading 0x0000000000000000

[process exited with code 1 (0x00000001)]

[LostRuins]

I don't have CUDA 12.5. Does this also happen with earlier versions of CUDA such as 11.7 and 12.1?

Also see if the official 1.66 release has any issues.

[jojorne]

Not working... Previously, I opened Kobo in my IDE. In there, it works. Now, I tried running it as I usually do, outside the IDE. It crashed. 🙃 It seems like there is something to do with Debug vs Release.

[pencilethics]

I am getting the above error with 1.66 and 1.65 on both CUDA 12.4.1 and 12.5 on a Release build on Visual Studio 2022 17.10.0; by explicitly installing the 17.9 toolset (MSVC 19.39), 1.66(.1) works again with CUDA 12.4.1. Yet to retest using CUDA 12.5, but at this stage I'm inclined to believe that that isn't the problem

[LostRuins]

Awesome thanks

[Djhutmose]

I had the same problem just recently, with CUDA 12.5.39 and Visual Studio 17.10.0. Only happened for Release builds, and Debug was fine. I resolved it by updating the msvcp140.dll and msvcp140_codecvt_ids.dll in the project root.

[jojorne]

I uninstalled Build Tool 14.39, then I built with 14.40 and executed:

del /q "cudart64_12.dll"
del /q "cudart64_110.dll"
del /q "msvcp140_codecvt_ids.dll"
del /q "msvcp140.dll"

This was enough to run on my end.