RuntimeError: CUDA error: no kernel image is available for execution on the device

[ykemiche]

Hi, I have cuda 11.7 on my system and I am trying to install functorch, since the stable version of pytorch for cuda 11.7 is not available here, I just run pip install functorch which also installs the compatible version of pytorch.

But when I run my code that uses the GPU, I get the following error :

RuntimeError: CUDA error: no kernel image is available for execution on the device

Is it possible to use functorch in my case?

[zou3519]

@ykemiche how did you install PyTorch with CUDA 11.7 support?

[ykemiche]

I followed this recommendation, I used the supported binary with CUDA 11.3 by runing this command : pip install --pre torch -f https://download.pytorch.org/whl/nightly/cu113/torch_nightly.html --upgrade

[zou3519]

Ah I see. My hypothesis is that your functorch version and pytorch version are incompatible. If you installed PyTorch via pip install --pre torch -f https://download.pytorch.org/whl/nightly/cu113/torch_nightly.html --upgrade, then you have a nightly version of PyTorch.

To install functorch with a nightly version of PyTorch you'd want to install functorch from source:

pip install ninja  # Makes the build go faster
pip install --user "git+https://github.com/pytorch/functorch.git"

[zou3519]

Can you please print out the following:

import torch
import functorch
print(torch.__version__)
print(torch.version.cuda)
print(functorch.__version__)

? Also, are you getting RuntimeError: CUDA error: no kernel image is available for execution on the device when you use only PyTorch and no functorch?

For some more context: functorch is CUDA agnostic (it doesn't have any CUDA-specific code), so it's most likely PyTorch that has problems with your CUDA setup.

[ykemiche]

That's what I did the first time, I've installed functorch on a venv from source as described on your Readme, by doing :

pip install --pre torch -f https://download.pytorch.org/whl/nightly/cu113/torch_nightly.html --upgrade
pip install ninja  # Makes the build go faster
pip install --user "git+https://github.com/pytorch/functorch.git"

I got a wheel building error as you can see on this picture :

After that, I started a new venv and directly installed functorch with pip install functorch but it seems that the torch version installed by default is incompatible with cuda 11.7.Here is by order, the output of prints :

1.12.1+cu102 10.2 0.2.1

And to answer to your last question, I get this error RuntimeError: CUDA error: no kernel image is available for execution on the device even though I'm not using functorch

[zou3519]

pip install functorch installs functorch 0.2.1 which is going to pull in PyTorch 1.12.1 with cu102 support. What you really want is to install PyTorch 1.12.1 with cu113 support (PyTorch's cu113 binary should work with CUDA 10.7). Could you please try the following?

pip install torch --extra-index-url https://download.pytorch.org/whl/cu113
pip install functorch

[ykemiche]

It worked, thanks for your help

[zou3519]

Glad to hear. We're working on making installing both pytorch and functorch easier. Please let us know if you have any other feedback.