search

Fugaku-LLM / DeepSpeedFugaku

Other
128 stars 7 forks source link

Request for Access to Specific Dynamic Libraries #3

Closed CaRoLZhangxy closed 5 months ago

CaRoLZhangxy commented 5 months ago

I am currently working with the repository and have encountered an issue where some dynamic libraries are missing access permissions, such as pytorch and Allreduce lib listed in the scripts. I am trying to apply some training acceleration techniques on this project, so these libraries are essential. Is there any possible way I can gain access to these dynamic libraries or source code?Thank you very much for your attention to this matter.

m-ymzk commented 5 months ago

Thank you for your question on this repository.

The pytorch we used was customized for the A64FX CPU from the Fujitsu repository. Also, there is no plan to release the source code for allreduce optimization specialized for TOFU network of Fugaku. So it would be good to use the pytorch environment that is best suited for each CPU and GPU.

Ref. https://github.com/fujitsu/pytorch/tree/r1.13_for_a64fx

CaRoLZhangxy commented 5 months ago

Got it. Thank you for your response!