search

mirage-project / mirage

A multi-level tensor algebra superoptimizer
https://mirage-project.readthedocs.io/
Apache License 2.0
341 stars 18 forks source link

Support for CUDA 12.2 #11

Open sam-h-bean opened 4 months ago

sam-h-bean commented 4 months ago

We are trying to deploy this image to a kubernetes cluster which does not have access to machines with CUDA 12.4 and when trying to run your demo I see

# python demo/demo_group_query_attention_spec_decode.py --checkpoint demo/checkpoint_group_query_attn_spec_decode.json
Cuda failure: 35
/usr/mirage/src/kernel/device_memory_manager.cu:30
Aborting...
python: /usr/mirage/src/kernel/device_memory_manager.cu:30: mirage::kernel::DeviceMemoryManager::DeviceMemoryManager(): Assertion `false' failed.

Curious if you have a way to run this on slightly older versions of CUDA. Will you be releasing newer docker images with support for different drivers?