Increase the startup time/make it configurable + look into HF_HUB_ENABLE_HF_TRANSFER for faster downloads

awslabs / fmbench-orchestrator

Run FMBench simultaneously across multiple Amazon EC2 machines to benchmark an FM across multiple serving stacks simultaneously

2 stars 2 forks source link

Open madhurprash opened 2 days ago

madhurprash commented 2 days ago

Extend the start up time from 1200 to 1500 (for aws silicon instances) + add HF_HUB_ENABLE_HF_TRANSFER for faster downloads of the model