flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving
https://flashinfer.ai
Apache License 2.0
768 stars 64 forks source link

cmake: fix DECODE_F8_DTYPES and DECODE_FP8_DTYPES discrepancy #287

Closed ibsidorenko closed 1 month ago

ibsidorenko commented 1 month ago

As a result of typo, it appeared two variables: DECODE_F8_DTYPES and DECODE_FP8_DTYPES in CMakeLists.txt. But it should be the same. This commit fixes this discrepancy and rename DECODE_F8_DTYPES -> DECODE_FP8_DTYPES.

ibsidorenko commented 1 month ago

cc @yzh119

ibsidorenko commented 1 month ago

Variable DECODE_FP8_DTYPES is used to generate fp8 kernels:

        # fp8 in, fp16 out
        foreach(dtype IN LISTS DECODE_FP8_DTYPES)
          set(generated_kernel_src ${PROJECT_SOURCE_DIR}/src/generated/single_decode_group_${group_size}_head_${head_dim}_layout_${kv_layout}_posenc_${pos_encoding_mode}_dtypein_${dtype}_dtypeout_f16.cu)
          add_custom_command(
            OUTPUT ${generated_kernel_src}
            COMMAND ${Python3_EXECUTABLE} ${PROJECT_SOURCE_DIR}/python/generate_single_decode_inst.py ${generated_kernel_src}
            DEPENDS ${PROJECT_SOURCE_DIR}/python/generate_single_decode_inst.py
            COMMENT "Generating additional source file ${generated_kernel_src}"
            VERBATIM
          )
          list(APPEND single_decode_kernels_src ${generated_kernel_src})
        endforeach(dtype)

But this variable was not set. Instead we set DECODE_F8_DTYPES. This commit fixes this discrepancy and rename DECODE_F8_DTYPES -> DECODE_FP8_DTYPES.