Is there any inference framwork to accelerate CuMo?

SHI-Labs / CuMo

CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts

Apache License 2.0

117 stars 8 forks source link

Closed leoozy closed 1 month ago

leoozy commented 1 month ago

Sorry to bother, does CuMo support batch inference?

chrisjuniorli commented 1 month ago

Not for now, but it's in our plans, stay tuned!

leoozy commented 1 month ago

Thanks