search

ggerganov / ggml

Tensor library for machine learning
MIT License
11.28k stars 1.05k forks source link

vulkan: fix build for GGML_VULKAN_RUN_TESTS, add TFLOPS to logging #961

Closed jeffbolznv closed 2 months ago

jeffbolznv commented 2 months ago

GGML_VULKAN_RUN_TESTS currently has build errors like:

ggml-vulkan.cpp(5251,5): error C2664: 'void ggml_vk_dispatch_pipeline(ggml_backend_vk_context *,vk_context &,vk_pipeline &,const std::initializer_list<vk::DescriptorBufferInfo> &,size_t,const void *,std::array<uint32_t,3>)': cannot convert argument 4 from 'initializer list' to 'const std::initializer_list<vk::DescriptorBufferInfo> &'
ggml-vulkan.cpp(5497,10): error C2039: 'staging': is not a member of 'ggml_backend_vk_context'

and runtime assertions like:

ggml-vulkan.cpp:2486: GGML_ASSERT(pipeline->descriptor_set_idx < pipeline->descriptor_sets.size()) failed

Fix these, and also add teraflops calculation to the matmul tests.

jeffbolznv commented 2 months ago

Ping for review.

I'm curious, is it better to do these sort of fixes in llama.cpp or in ggml?

ggerganov commented 2 months ago

Sorry for the delay. Pinging @0cc4m as they are the main maintainer of the vulkan backend. If we don't get a response in a day or two, will try to review myself - please ping again if necessary.

I'm curious, is it better to do these sort of fixes in llama.cpp or in ggml?

llama.cpp has more eyes in general, so chances for review are better. Although some of the Vulkan PRs are also stale over there (https://github.com/ggerganov/llama.cpp/pull/9407).