performance problem about internvl image embedding using ggml.dll

[cjsdurj]

problem desc

Image embedding using ggml.dll provided by ipex will become slower and slower, while using llama.cpp a1631e5 build performance is stable.

• ipex-llm
• llama.cpp a1631e5
  

test code

clip source code can be found in https://github.com/ggerganov/llama.cpp/pull/9403

#include "clip.h"

#include "internvl.h"
#include "iostream"

int main(int argc, char* argv[]) {
  std::string model_path;
  std::string image_path;
  std::string device;

  for (int i = 1; i < argc; i += 2) {
    std::string arg = argv[i];
    if (arg == "--model") {
      model_path = argv[i + 1];
    } else if (arg == "--image") {
      image_path = argv[i + 1];
    } else if (arg == "--device") {
      device = argv[i + 1];
    }
  }

  auto ctx_clip = clip_model_load(model_path.c_str(), 1 ,device);

  for (int i = 0; i < 20; i++) {
    auto embed = internvl_image_embed_make_with_filename(ctx_clip, 4,
                                                         image_path.c_str());
    std::cout << embed->embed[0] << "\n";
  }
  return 0;
}

env

ultra 7 155H igpu , windows11

[rnwang04]

Hi @cjsdurj , thanks for pointing out this issue. I have fixed it, you could try it again with ggml.dll released in pip install ipex-llm>=2.2.0b20241111 tomorrow.