FileNotFoundError: Could not find module [...] koboldcpp_cublas.dll

[ImNotAlexander]

After a system rebuild and fresh Windows install recently I reinstalled all my programs only to find koboldcpp has a problem where it apparently can't find the correct file in the temp directory it creates. Running the exe normally leads to a crash upon pressing launch with CuBLAS enabled- and only with that enabled- and running from the command line to see what's going on as the FAQ instructs gives this:

E:\koboldcpp>"koboldcpp 1.62.2.exe" --usecublas 1 --gpulayers 15 --model mythomax-l2-13b.Q5_K_S.gguf
***
Welcome to KoboldCpp - Version 1.62.2
Attempting to use CuBLAS library for faster prompt ingestion. A compatible CuBLAS will be required.
Initializing dynamic library: koboldcpp_cublas.dll
Traceback (most recent call last):
  File "PyInstaller\loader\pyimod03_ctypes.py", line 53, in __init__
  File "ctypes\__init__.py", line 373, in __init__
FileNotFoundError: Could not find module 'C:\Users\xxx\AppData\Local\Temp\_MEI89242\koboldcpp_cublas.dll' (or one of its dependencies). Try using the full path with constructor syntax.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "koboldcpp.py", line 3195, in <module>
  File "koboldcpp.py", line 2878, in main
  File "koboldcpp.py", line 270, in init_library
  File "PyInstaller\loader\pyimod03_ctypes.py", line 55, in __init__
pyimod03_ctypes.PyInstallerImportError: Failed to load dynlib/dll 'C:\\Users\\xxx\\AppData\\Local\\Temp\\_MEI89242\\koboldcpp_cublas.dll'. Most likely this dynlib/dll was not found when the application was frozen.
[2008] Failed to execute script 'koboldcpp' due to unhandled exception!

After some Googling, I found one other result that stood out, where a user had a similar result from a different error: https://github.com/LostRuins/koboldcpp/issues/599 Their problem turned out to be their CPU not supporting AVX2, which is a cause that has shown up in other results- however, while my GPU (GTX 1060 6GB, Nvidia drivers 551.86) is outdated, it does still support CUDA and my CPU (i5-12600KF) is certainly new enough to support AVX2, as did the old system it replaced. The system info line during a successful launch reads:

System Info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | FMA = 1 | NEON = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 |

Assuming that maybe having a non-standard Windows install (Win10 Enterprise LTSC) could be causing problems, I tried all other options and went back to older releases of koboldcpp to see where the error may have began:

• On the latest release, 1.62.2, 'Use CuBLAS' fails to launch with the aforementioned error. OpenBLAS, CLBlast, Vulkan, and No BLAS all work as expected including GPU loading where applicable, when run with the visual interface or directly from the command line. The four Old CPU options including failsafe mode also work, albeit at a glacial pace (which is to be expected).
• When ran with one of the other options, "koboldcppcublas.dll" does exist in the numbered temp folder that's created, alongside the other koboldcpp*.dll files. I don't know if it exists when ran with Use CuBLAS selected as koboldcpp exits and deletes the folder immediately.
• The version of koboldcpp I had backed up, 1.54, does still work correctly with Use CuBLAS selected. No errors or hangs and runs at the expected speed when processing or generating tokens. When this or other working versions are running, the temp folder again includes the correct .dll and finds it just fine.
• The newest version of koboldcpp that allows me to use CuBLAS is 1.59.1, which again behaves as expected. Version 1.60.1 and newer all fail in the same manner as listed above with only the line number in the traceback varying by version, and only when trying to run with Use CuBLAS selected, regardless of other options.
• Each version was tested to actually 'work' (and not just load a model) with both the embedded Kobold Lite browser page and with Sillytavern connected after, and using two different models (one old that I knew worked, mythomax-13b, and one new, Mixtral-8x7b, which should be supported since even earlier than 1.54 according to the release notes).
• The issue persists regardless of what the exe is named- the above log is only the most recent one after renaming koboldcpp.exe to include the version number as I was getting a little lost juggling six of them around. Renaming it to koboldcpp.exe or running a fresh download doesn't change or fix anything I can see.

Frankly I'm at a loss here and I'm left scratching my head, so if there's any extra information that would help I'd be glad to provide it.

[LostRuins]

Okay, I think this is likely caused by a missing runtime dll on your system. The strange thing is that no new cuda related dlls were added between the abovementioned versions.

Here's what you can try:

• Obtain the Dependencies tool at https://github.com/lucasg/Dependencies/releases/download/v1.11.1/Dependencies_x64_Release.zip
• Run the tool DependiencesGUI.exe
• Run the latest koboldcpp.exe and let the GUI launch but do not select any backend.
• Using the DependenciesGUI tool, navigate to the numbered Temp Folder (Usually labelled starting with MEI_). Find the koboldcpp_cublas.dll file and open it.
• Observe what libraries are found or missing. If possible, take a screenshot and share it here.

It should look like this:

This will help narrow down what dependencies you are missing.

[ImNotAlexander]

Can do.

Everything seems to be in order (although some are listed in my temp folder, not in system32) other than MSVCP140_CODECVT_IDS.dll.

[LostRuins]

Yep that is the culprit. MSVCP140_CODECVT_IDS.dll is missing. I'll include that runtime into the next version which should hopefully solve the issue.

For now, can you try the CI build at this link: https://github.com/LostRuins/koboldcpp/actions/runs/8649876498/artifacts/1405863309

[LostRuins]

Another option is manually installing the VC redist packages (e.g. here https://learn.microsoft.com/en-US/cpp/windows/latest-supported-vc-redist?view=msvc-170)

[ImNotAlexander]

I tried the build you linked and it works great so far- loads models, processes and generates all with CuBLAS enabled just fine. Thank you for the help and the speedy explanation and patch.