能否支持多卡推理

thunlp / InfLLM

The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory"

MIT License

269 stars 21 forks source link

Closed cat-sun closed 5 months ago

guyan364 commented 5 months ago

您好，目前已经支持多进程的 benchmark，可以查看 scripts/multiprocessing-benchmark.sh chat 的多卡推理目前没有计划支持