search

FMInference / FlexLLMGen

Running large language models on a single GPU for throughput-oriented scenarios.
Apache License 2.0
9.18k stars 548 forks source link

interesting you can crop 65b #103

Open seoeaa opened 1 year ago

seoeaa commented 1 year ago

interesting you can crop 65b https://neuralmagic.com/blog/sparsegpt-remove-100-billion-parameters-for-free/