Exception creating TF Session with CUDA: "CUDA driver version is insufficient for CUDA runtime version"

[vpenades]

When I try to create a TF Session with CUDA, I get this error:

"CUDA driver version is insufficient for CUDA runtime version"

Now, obviously the solution is to update my drivers... but that's not the problem.

The problem is that both Emgu.CV.Cuda.CudaInvoke.HasCuda and TF.TfInvoke.IsGoogleCudaEnabled return TRUE

Now this is the problem: if we're to deliver a software in the wild, we have to ensure the client's machine has the right hardware and software, which means we need a way to detect it has the right cuda drivers, among other things.

The application workflow should go like this:

1. Do we have the hardware and drivers ready?
   • NO: Report to the user what's missing and EXIT.
   • YES: Continue
2. Create TF.Session
3. Start VideoCapture
4. Loop

Trusting IsGoogleCudaEnabled in step (1) is clearly useless because it will let the application continue running until step (4) at which point it will throw an exception and crash the application.

So we need to know before creating a Session that we absolutely meed all the hardware and driver requirements, I would suggest something like this:

enum CudaStatus
{
    Ready, DriversNotFound, DriversInsufficientVersion , FatalError, AnyOtherUsefulErrorThatCanHintTheEndUserHowToFixProblem
}

CudaStatus TF.TfInvoke.GetGoogleCudaStatus();

Finally, I decided to update my nVidia drivers, which come with Cuda 11, and it seems it's the next big iteration, and tensorflow needs Cuda 10.2 .... so tensorflow is throwing exceptions when running inferences.