search

StanfordLegion / legion

The Legion Parallel Programming System
https://legion.stanford.edu
Apache License 2.0
657 stars 146 forks source link

Realm: CUDA 12.3 device kernel image is invalid #1690

Open syamajala opened 1 month ago

syamajala commented 1 month ago

I am trying to run S3D with CUDA 12.3. I did a fresh checkout and am seeing this at start up:

[1 - 155555514000]    0.000000 {5}{gpu}: /lustre/scratch/vsyamaj/legion_s3d_nscbc/legion/runtime/realm/cuda/cuda_module.cc(3463):CUDA_DRIVER_FNPTR(cuModuleLoadDataEx)( &device_module, realm_fatbin, 0, NULL, NULL) = 200(CUDA_ERROR_INVALID_IMAGE): device kernel image is invalid
syamajala commented 1 month ago

CUDA driver needs to be updated on the machine.

eddy16112 commented 1 month ago

reopen this issue because we plan to add a driver version check in realm to provide a clearer error message.