search

pentium3 / sys_reading

system paper reading notes
236 stars 12 forks source link

Training 175B Parameter Language Models at 1000 GPU scale with Alpa and Ray #257

Open pentium3 opened 1 year ago

pentium3 commented 1 year ago

https://www.anyscale.com/blog/training-175b-parameter-language-models-at-1000-gpu-scale-with-alpa-and-ray?utm_content=242470147&utm_medium=social&utm_source=linkedin&hss_channel=lcp-28464994

pentium3 commented 1 year ago

https://www.anyscale.com/blog/ray-common-production-challenges-for-generative-ai-infrastructure