Is bfloat16 going to be supportted?

ColfaxResearch / cutlass-kernels

MIT License

137 stars 25 forks source link

Is bfloat16 going to be supportted? #11

Open hxdtest opened 4 weeks ago

hxdtest commented 4 weeks ago

 // For now, only half_t is supported. TF32 is WIP.
  // Instantiate the function template for different HEADDIMS.
  // For now, only half_t is supported. TF32 is WIP.
  if (kHeadSize == 64) {
    testFmhaForward<cutlass::half_t, 64>(seqLength, seqLength, numHeads,
                                         batchSize, iterations, refCheck,
                                         printValues, nStreams);

Why only half_t is supported?

jayhshah commented 4 weeks ago

Only for simplicity, you can change the template parameter yourself if you want to. However I recommend using FlashAttention-3 instead now.