RuntimeError CUDA error: no kernel image

[pribadihcr]

Hi, When running the sample code I got the following error

 return causal_conv1d_cuda.causal_conv1d_update(
RuntimeError: CUDA error: no kernel image is available for execution on the device

I did install pip install causal--conv1d and using gpu gtx-1080

big thanks

[krasin]

It appears that the minimum requirement for running Cobra is RTX 30 series. While it's a separate issue, today, I tried to run Cobra on RTX 2080, and it really wanted a proper bfloat16 support, which RTX 20 series lack.

[h-zhao1997]

@pribadihcr We have currently only attempted testing on the A100 and A800, and have not yet tested support for older GPU architectures.

@krasin Similarly, I'm not sure if the RTX 20 series GPU will work, but I made a simple modification to support inference at torch.float16 precision. You could test this to see if it resolves your issue.

[krasin]

Thank you @h-zhao1997 . This is to confirm that the latest revision of the repository works fine on RTX 20 series. Specifically, I tested on RTX 2070 and CUDA 12.2:

$ nvidia-smi
Thu Mar 28 08:43:30 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.129.03             Driver Version: 535.129.03   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 2070        On  | 00000000:02:00.0 Off |                  N/A |
| 29%   30C    P8               8W / 175W |      1MiB /  8192MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

And here is the output from scripts/generate.py:

$ python ./scripts/generate.py
=> Default Image URL: `https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/beignets-task-guide.png`
===

|=>> Enter (i)mage to fetch image from URL, (p)rompt to update prompt template, (q)uit to exit, or any other key to enter input questions: 

[*] Entering Chat Session - CTRL-C to start afresh!
===

|=>> Enter Prompt: describe the picture
        |=>> VLM Response >>> A table with various food items and drinks, including coffee, powdered sugar donuts, and water glasses.

Big thanks!