Failing to upgrade to 0.4.0 version

[asit2898]

I am currently using 0.3.16 version. To upgrade to 0.4.0 I tried the following:

$ pip install --upgrade deepspeed

This results in :

Successfully built deepspeed
Installing collected packages: deepspeed
  Attempting uninstall: deepspeed
    Found existing installation: deepspeed 0.3.16
    Uninstalling deepspeed-0.3.16:
      Successfully uninstalled deepspeed-0.3.16
Successfully installed deepspeed-0.4.0

However, when I try importing deepspeed I get the following error:

$ python3
>>> import deepspeed

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jupyter/libraries/nb_env/lib64/python3.6/site-packages/deepspeed/__init__.py", line 41, in <module>
    __version_major__, __version_minor__, __version_patch__ = _parse_version(__version__)
  File "/home/jupyter/libraries/nb_env/lib64/python3.6/site-packages/deepspeed/__init__.py", line 36, in _parse_version
    return ver.major, ver.minor, ver.micro
AttributeError: 'Version' object has no attribute 'major'

I get a similar error for ds_report:

$ ds_report

Traceback (most recent call last):
  File "/home/jupyter/libraries/nb_env/bin/ds_report", line 3, in <module>
    from deepspeed.env_report import main
  File "/home/jupyter/libraries/nb_env/lib64/python3.6/site-packages/deepspeed/__init__.py", line 41, in <module>
    __version_major__, __version_minor__, __version_patch__ = _parse_version(__version__)
  File "/home/jupyter/libraries/nb_env/lib64/python3.6/site-packages/deepspeed/__init__.py", line 36, in _parse_version
    return ver.major, ver.minor, ver.micro
AttributeError: 'Version' object has no attribute 'major'

Also attaching the output of pip list --local:

$ pip list --local
Package             Version
------------------- ---------------------
apex                0.1
appdirs             1.4.4
backcall            0.1.0
cfgv                3.2.0
clang-format        9.0.0
commonmark          0.9.1
cpufeature          0.1.1
dataclasses         0.7
datasets            1.7.0
decorator           4.3.0
deepspeed           0.4.0
dill                0.3.3
distlib             0.3.1
editdistance        0.5.3
filelock            3.0.12
fsspec              0.8.7
future              0.18.2
huggingface-hub     0.0.8
identify            1.5.9
imageio             2.9.0
importlib-resources 3.3.0
ipykernel           4.8.2
ipynb-py-convert    0.4.6
ipython             6.3.1
ipython-genutils    0.2.0
jedi                0.12.0
joblib              1.0.1
jupyter-client      5.2.3
jupyter-core        4.4.0
lmdb                1.0.0
multiprocess        0.70.11.1
natsort             7.0.1
ninja               1.10.0.post2
nodeenv             1.5.0
numpy               1.19.4
parso               0.2.0
pexpect             4.5.0
pickleshare         0.7.4
Pillow              8.0.1
pip                 20.2.4
pre-commit          2.8.2
prompt-toolkit      1.0.15
ptyprocess          0.5.2
pyarrow             3.0.0
Pygments            2.2.0
pytest-forked       1.3.0
python-dateutil     2.8.1
PyWavelets          1.1.1
pyzmq               17.0.0
recommonmark        0.6.0
regex               2020.11.13
sacremoses          0.0.43
scikit-image        0.17.2
setuptools          39.0.1
simplegeneric       0.8.1
six                 1.11.0
sphinx-rtd-theme    0.5.0
tensorboardX        1.8
thop                0.0.31.post2005241907
tifffile            2020.9.3
tokenizers          0.10.1
toml                0.10.2
torch               1.6.0+cu92
torchinfo           0.1.1
torchvision         0.7.0+cu92
tornado             5.0.2
tqdm                4.49.0
traitlets           4.3.2
transformers        4.6.1
typing-extensions   3.7.4.3
utils               1.0.1
virtualenv          20.1.0
wcwidth             0.1.7
wheel               0.31.0
xxhash              2.0.0
WARNING: You are using pip version 20.2.4; however, version 21.1.2 is available.
You should consider upgrading via the '/home/jupyter/libraries/nb_env/bin/python3.6 -m pip install --upgrade pip' command.

[jeffra]

Hmm strange. We did recently add packaging to our requirements.txt and i don't see it in your pip list. Can you try installing it? Maybe the upgrade for some reason isn't triggering the new requirement.

https://github.com/microsoft/DeepSpeed/blob/aa16828c267dc7a790a5cc2f43f93a67617a2364/requirements/requirements.txt#L8

[asit2898]

@jeffra Thanks for pointing out the packaging library. It was installed in the parent environment and hence did not show up in pip list --local. This seems to be an issue with the 16.8 version of packaging. After upgrading to 20.9 the issue got resolved!