Batch inference support (self-assigning)

Blaizzy / mlx-vlm

MLX-VLM is a package for running Vision LLMs locally on your Mac using MLX.

MIT License

166 stars 14 forks source link

Open willccbb opened 3 weeks ago

willccbb commented 3 weeks ago

Would be nice to have batch inference support similar to mlx_parallm, happy to try and add soon. @Blaizzy can you assign this to me?

Blaizzy commented 3 weeks ago

Hey Will,

Yes, that would be awesome!

I have assigned the task to you 😀

Blaizzy commented 3 weeks ago

Please comment #40 here so I can assign it to you and discuss details on a single issue.