search

daadaada / turingas

Assembler for NVIDIA Volta and Turing GPUs
MIT License
201 stars 40 forks source link

Request for help regarding the cubin format #11

Open eyalroz opened 2 years ago

eyalroz commented 2 years ago

Hello @daadaada ,

I wanted to write you an email, but couldn't find an address anywhere so I'm filing an issue instead.

I'm the author of eyalroz/cuda-api-wrappers. I'm interested in allowing users of my wrappers library to be able to check which targets cubin files contain code for. Unfortunately, the CUDA Driver API itself does not have API functions to check this.

Now, it seems - especially when leafing through your code here - that cubin's are basically ELF-like. But - they're not properly ELF, in the sense that readelf refuses to read them and give information about them.

I'm not an ELF expert (quite the opposite), and the code in cubin.py has a lot of magic numbers and I can't make heads and tails of it. I would like to ask for your help in understanding how, programmatically, to open a cubin file and determine which kernels it holds for which targets/architectures. Or at least - what's the format in which this information is stored relative to ELF in general.

daadaada commented 2 years ago

Hi @eyalroz,

I think the target information is embedded in the flag field of the ELF header. (https://github.com/daadaada/turingas/blob/master/turingas/cubin.py#L46). So you can get the target arch by reading the flag field of the ELF header with readelf -h xxx.cubin

The lower 8 bits of the Flag field represent the target arch. For example, if Flags is 0x4b054b, the target arch is 0x4b (75).