search

monatis / clip.cpp

CLIP inference in plain C/C++ with no extra dependencies
MIT License
415 stars 24 forks source link

Implement bicubic interpolation #74

Open monatis opened 9 months ago

monatis commented 9 months ago

Currently clip.cpp uses linear interpolation in image preprocessing. The original implementation uses the bicubic interpolation from Pillow. It needs refactoring from Pillow https://github.com/python-pillow/Pillow/blob/main/src/libImaging/Resample.c#L46-L62

lithdew commented 6 days ago

An implementation of bicubic interpolation is available inside llama.cpp.

https://github.com/ggerganov/llama.cpp/blob/master/examples/llava/clip.cpp#L1431

monatis commented 6 days ago

Yes, I upstreamed clip.cpp to llama.cpp for LLaVA, and now there's some functionality that needs to be downstreamed from llama.cpp back to this repo. Currently I don't have enough time to maintain it, though. Maybe next month.

lithdew commented 6 days ago

I'm looking to use clip.cpp in a project soon - would you mind if I make a PR that brings some of the features in llama.cpp into clip.cpp?

monatis commented 6 days ago

@lithdew That would be awesome. And feel free to ping me if you need to ask anything about the implementation.