Custom batch decoding algorithm

anarchy-ai / LLM-VM

irresponsible innovation. Try now at https://chat.dev/

https://anarchy.ai/

MIT License

465 stars 150 forks source link

Custom batch decoding algorithm #368

Open VictorOdede opened 8 months ago

VictorOdede commented 8 months ago

Implement parallel decoding for any model: https://arxiv.org/abs/2305.10427

rishirishhh commented 7 months ago

assign it to me please

mmirman commented 7 months ago

@rishirishhh you are welcome to work on it! Getting assigned tickets officially requires contributor status - which you can achieve by winning bounties!

bilal-aamer commented 7 months ago

Hey @VictorOdede, would like to take a shot at this issue