Closed Xwmiss closed 1 month ago
Even I change the input_raw in the function "encode_image_with_clip" to make the input for torch and llama.cpp is must same, but the result of these two inference way is still different and the Cosine Similarity is still 0.75.
And the another thing is need to say, while the result of siglip+resampler is different, but the final llm result for the image and question is right, which make me confused more....
@all Hi, I don't always pay attention to the issue area in this fork code repo.
Hi, thanks for your work firstly. while I find that I use the same picture as input of torch_model and llama.cppmodel, the res(1, 96, 4096) of siglip+resampler part is different, the Cosine Similarity is only 0.75. I am confused now.