search

Rust-GPU / Rust-CUDA

Ecosystem of libraries and tools for writing and executing fast GPU code fully in Rust.
Apache License 2.0
2.97k stars 112 forks source link

Passing fp16 buffers #113

Closed srush closed 10 months ago

srush commented 10 months ago

I was curious what the state of passing in fp16 buffers was in Cust? I'm not trying to do anything fancy, just pass in fp16 to an existing PTX function not written in Rust. Can this be done currently with half or similar libraries? Do I need to encode this myself?

RDambrosio016 commented 10 months ago

You can do this with half, or you can to_bits() it into u16, or pass it as a byte slice, the kernel launching does not check if parameter types are right, so it doesn't really care what the input is, as long as it turns into the expected bytes.

srush commented 10 months ago

If any one else is curious, I ended up doing it with half / newtype. 

#[derive(Default, Clone, Copy, Debug)]
struct F16(f16);
unsafe impl DeviceCopy for F16 {}
...        
let t: [F16; OUT] = [F16(f16::from_f32(0.0)); OUT];
t.as_dbuf()?,

Works great!