CudaBlas Handler Initialization Error

[chenw11]

Hi Kunzmi, This maybe a very basic question. I encountered an error when trying to initializing a cublas handle using the CudaBlas wrapper. I ran into the following error when calling the cublas constructor, where it points out that the cublas64_11.dll from cuda toolbox is missing. So far, I have tried downloading the latest ManagedCuda (11.1) from GitHub repository, build as 64-bit in local, then import the CudaBlas.dll (8.0.13) to my application reference. It gives a similar error but with different missing cublas dll file (cublas64_80.dll). Also I tried to download the CudaBlas packages (11.0.43) from NUGet repository, and installed using NUGet package installer. Unfortunately, it still give me the same error.

Yet, the Managedcuda simpleCUBLAS example is working just fine. The only difference I notice so far comparing with my application, is the .NET Framework is at 4.5.2, while mine is at 4.6.2. But after building my app in 4.5.2, the error still remain.

Obviously, base on the error message, this file "cublas64_11.dll" is missing when the wrapper is calling the native function. I'm able to locate this file at the NVDIA Toolkit folder, but since it is a Cuda C/C++ dll, I can't directly add it to my C# reference. Any insights from you would be much appreciated!

[kunzmi]

Given that the sample is running, I bet that your application is running in 32-bit and can't load the 64-bit cublas-DLL. If you are in Any CPU-mode, check in the project settings that the prefer 32-bit option is not set. In case you're in x86-32-bit mode, well change to x64.

[chenw11]

Thanks for the quick response. In fact, I believe my app is already ran in 64-bit, and below is the config setting from the Build page, which is basically the same as the example's. In the CUBLAS Nuget wrapper description, you mentioned that it is compatible with .net Core 3.x. I don't know if this is the correct way to check my donet version, but base on the command, my .net version is "2.1.52", Not sure if this could cause this problem.

[kunzmi]

Then it can only be that your PATH environment variable doesn't contain the correct path to the CUDA libraries, e.g. C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin; (or that the corresponding CUDA toolkit is not installed, but from your screenshot it seems to be there). When you do echo %PATH% in a console, check that the CUDA toolkit version you want to use actually appears, also that the PATH variable content is not too long, I think there's a limit of characters...

[chenw11]

You are right it looks like the PATH doesn't contain any CUDA environment variable, so I added v_8, v_11, and v11_1 in the system property setting. But it is not shown upon echo %PATH% However, as the PATH shown. CUDA V_8 is in place, but I still got the same error when cublas (v8.0.13) is called.

[chenw11]

Just want to share some new finding here. I was able to replicate this similar error by replacing the CUFFT.dll in my app from 11.0.43 to 8.0.13 and build with 11.0.43 ManagedCuda. So it seems like it is related to dll build version compatibility with the main ManagedCuda dll.

Besides, I can initialize the cublas constructor just fine by building a new test console app with 8.0.13 ManagedCUDA and 8.0.13 CudaBlas. This is exactly the same build environment as SimpleCuBLAS example's.

But when I build the test console app with 11.0.43 ManagedCUDA and 11.0.43 CudaBlas, the error will occur. Do you have a new version of Cudabls that maybe you could try to see if it has the same problem?

[kunzmi]

Hi, I'm a bit confused of what versions you actually exchanged and tested, but managedCuda and CudaBlas (the wrapper for cublas) must be of the same version targeting one specific CUDA toolkit version. If all are from the same version, it should run smoothly. The .net framework shouldn't have any influence on this. But looking at the length of your %PATH% variable, are you sure that this is less then 2048 characters? (If I remember correctly that is the maximum allowed length on a Windows system). A simple test for this is, if you just copy the original Nvidia DLL of the correct version just directly in the folder of application together with the managedCuda DLLs. If it then runs, then your PATH is messed up.

[chenw11]

Hi Kunzmi, Thank you very much for your insight. I have gone ahead to check the length of my PATH and it is less than 2048 (otherwise it won't let me add more variables). I even move the NVDIA CUDA 11.0 path to the first place but it still doesn't help to resolve this problem. I also found "cublas64_11.dll" is already in my bin folder, but when my app runs it still couldn't find it.

Here is a screenshot of the error details you might find helpful

Here is my cuda file compile comand

[chenw11]

I want to share a good news after weeks of debugging. This BUG is resolved if I initialize the cublashandle in my Main program's constructor. Since I'm developing a Window Form Application, so it I'm initializing this object in the Main{}scope. However, this is a dummy handle that will not be used in the main class though, because I'm using a separated processing object with a different cublas handle to handle all the GPU codes. But to avoid this "cublas64_11.dll" missing error, at least this is the only work around I found. I hope this help the others in the future.