Wrong import in zero - Githubissues

EleutherAI / oslo

OSLO: Open Source for Large-scale Optimization

https://oslo.eleuther.ai

174 stars 29 forks source link

Wrong import in zero #169

Closed jinwonkim93 closed 1 year ago

jinwonkim93 commented 1 year ago

Title

Prevent from using torch 2.0

Description

Some of feature have changed in torch 2.0. and oslo has dependency on torch._six which no longer support by torch 2.0.

olso Dependency

https://github.com/EleutherAI/oslo/blob/910c789e7f46d2876b964c221d31984b7924974f/oslo/torch/nn/parallel/data_parallel/zero/sharded_optim/_utils.py#L19

other issues

https://github.com/microsoft/DeepSpeed/issues/2845

Linked Issues

resolved #00

KKIEEK commented 1 year ago

I think the following solution seems better. https://github.com/hpcaitech/ColossalAI/commit/8e3f66a0d1ab44ecbee5917d15320e33ad118b5f

jinwonkim93 commented 1 year ago

I think the following solution seems better. hpcaitech/ColossalAI@8e3f66a

you are right. what do you think of limiting python 2.0 until tested.

KKIEEK commented 1 year ago

In my opinion, supporting PyTorch 2.0 is very important since PyTorch 2.0 support powerful new features such as torch.compile. Therefore, it would be better to solve versioning issues whenever the issue arises.