search

flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving
https://flashinfer.ai
Apache License 2.0
1.46k stars 143 forks source link

Fix compile error of OptionalCUDAGuard and device_of #613

Closed reyoung closed 5 days ago

reyoung commented 6 days ago

There are some compile errors in the main branch, like

/app/python/csrc_aot/single_prefill.cu(59): error: namespace "at::cuda" has no member "OptionalCUDAGuard"
    const at::cuda::OptionalCUDAGuard device_guard(device_of(device));
                    ^

/app/python/csrc_aot/single_prefill.cu(59): error: identifier "device_of" is undefined
    const at::cuda::OptionalCUDAGuard device_guard(device_of(device));

cc @yzh119

zhyncs commented 5 days ago

Thanks for this fix! ref https://github.com/flashinfer-ai/flashinfer/pull/611 I’ll help add build workflow these days. cc @yzh119