search

FMInference / FlexLLMGen

Running large language models on a single GPU for throughput-oriented scenarios.
Apache License 2.0
9.22k stars 549 forks source link

Suggestion: Add GPT-NeoX 20B support #9

Closed ElleLeonne closed 1 year ago

ElleLeonne commented 1 year ago

JAX is already a library that is optimized for GPU training, and the NeoX repo itself already requires significant GPU resources that could benefit from offloading.