[BOUNTY - $100] Vision Model Integration Test

exo-explore / exo

Run your own AI cluster at home with everyday devices 📱💻 🖥️⌚

GNU General Public License v3.0

14.78k stars 791 forks source link

[BOUNTY - $100] Vision Model Integration Test #328

Open AlexCheema opened 1 month ago

AlexCheema commented 1 month ago

As expected since we don't have tests for vision models, they broke unexpectedly: https://github.com/exo-explore/exo/issues/327
Create a test similar to the existing one (chatgpt_api_integration_test_mlx) we have for text in .circleci/config.yml

reddyn12 commented 1 month ago

I can do this

AlexCheema commented 1 month ago

I can do this

go go go!!

varshith15 commented 1 month ago

@AlexCheema this is already done in my pixtral PR (https://github.com/exo-explore/exo/pull/218/files#diff-3254d7af7026ebfe4e98753ce878db27e9d55796bec2744632a5633f4ad6a841), ill just move them to run on full pipeline.

I'm half way through with llama visual model as well, will raise a combined PR by EOW

reddyn12 commented 1 month ago

Looks like @varshith15 got it. Ill go back to tinygrad llava

AlexCheema commented 1 month ago

@AlexCheema this is already done in my pixtral PR (https://github.com/exo-explore/exo/pull/218/files#diff-3254d7af7026ebfe4e98753ce878db27e9d55796bec2744632a5633f4ad6a841), ill just move them to run on full pipeline.

I'm half way through with llama visual model as well, will raise a combined PR by EOW

Wow, awesome! Okay no need then! This bounty is yours, sir!

AlexCheema commented 1 month ago

@AlexCheema this is already done in my pixtral PR (https://github.com/exo-explore/exo/pull/218/files#diff-3254d7af7026ebfe4e98753ce878db27e9d55796bec2744632a5633f4ad6a841), ill just move them to run on full pipeline.

I'm half way through with llama visual model as well, will raise a combined PR by EOW

I was also hoping for a circleci integration test here.

varshith15 commented 1 month ago

oh ok, that would be a very small change/addition right? - will do that