How to estimate the maximum context length this repo can support for larger models?

jzhang38 / EasyContext

Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.

Apache License 2.0

653 stars 47 forks source link

How to estimate the maximum context length this repo can support for larger models? #42

Open JingyangDeng opened 4 months ago

JingyangDeng commented 4 months ago

I want to fine-tune Qwen2-72B using my data. Could you please estimate how long a context this repo can support on a setup with 16 or 32 A100/A800 (80G) GPUs? Thanks!