Closed ibsidorenko closed 1 month ago
cc @yzh119
Variable DECODE_FP8_DTYPES is used to generate fp8 kernels:
# fp8 in, fp16 out
foreach(dtype IN LISTS DECODE_FP8_DTYPES)
set(generated_kernel_src ${PROJECT_SOURCE_DIR}/src/generated/single_decode_group_${group_size}_head_${head_dim}_layout_${kv_layout}_posenc_${pos_encoding_mode}_dtypein_${dtype}_dtypeout_f16.cu)
add_custom_command(
OUTPUT ${generated_kernel_src}
COMMAND ${Python3_EXECUTABLE} ${PROJECT_SOURCE_DIR}/python/generate_single_decode_inst.py ${generated_kernel_src}
DEPENDS ${PROJECT_SOURCE_DIR}/python/generate_single_decode_inst.py
COMMENT "Generating additional source file ${generated_kernel_src}"
VERBATIM
)
list(APPEND single_decode_kernels_src ${generated_kernel_src})
endforeach(dtype)
But this variable was not set. Instead we set DECODE_F8_DTYPES
. This commit fixes this discrepancy and rename DECODE_F8_DTYPES
-> DECODE_FP8_DTYPES
.
As a result of typo, it appeared two variables:
DECODE_F8_DTYPES
andDECODE_FP8_DTYPES
in CMakeLists.txt. But it should be the same. This commit fixes this discrepancy and renameDECODE_F8_DTYPES
->DECODE_FP8_DTYPES
.