search

microsoft / CNTK

Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit
https://docs.microsoft.com/cognitive-toolkit/
Other
17.49k stars 4.3k forks source link

Requested CNTK has has different version in metadata: '2.7' #3834

Closed jialinyi94 closed 3 years ago

jialinyi94 commented 3 years ago

I was trying to install CNTK on my Linux desktop. The environment is Python 3.6, Ubuntu 16.04 LTS, CPU AMD Ryzen 5 2600 Six-Core Processor × 12 and GPU GeForce GTX 1060 6GB/PCIe/SSE2 with 64-bit.

Following the instructions here, I run

$ pip install cntk-gpu

it downloaded a wheel file but cannot install it. The error message goes:

Collecting cntk-gpu Using cached cntk_gpu-2.7.post1-cp36-cp36m-manylinux1_x86_64.whl (522.7 MB) ERROR: Requested cntk-gpu from https://files.pythonhosted.org/packages/c2/50/9bae3b117e1131ddfafadb2beceb9d344e9b5f8aab97a38a7558ef7f15b3/cntk_gpu-2.7.post1-cp36-cp36m-manylinux1_x86_64.whl#sha256=495ec2b5abc52fb7496f5c20bbc119dff18a29f0f65cb87f2cbb8f78153703f6 has different version in metadata: '2.7'

I searched the solutions but cannot find one. Thank you for your patience and help.

thiagocrepaldi commented 3 years ago

I see that you are using a locally cached wheel package, hopefully this is jus an older version that was fixed already

However, a possible quick hack is to download the wheel from the URL yo mentioned and rename the .whl file so that .post1 is removed from the filename. and do a pip install using the full path of the local whl package

The conflict seems to be 2.7 vs 2.7.post1 in the name

cwchung85 commented 3 years ago

Hi @thiagocrepaldi , Thanks for the info! We ran into the same problem in our CICD pipeline. So what do we need to do to get the fix? (We cannot do something manually in CICD).

Here are some additional notes:

  1. It used to work weeks ago, and just failed today when we triggered a new build. The problem appeared in cntk 2.7 as well as cntk-gpu 2.7, both for python=3.6. We tried to install both for ubuntu docker images.
  2. We are behind a corporate firewall. We use artifactory channel to access/download the packages
  3. In my own manual test of this issue on a Linux RHEL7, I was able to repro the problem by pip install --no-cache-dir cntk. (Prior to that, I didn;t use --no-cache-dir and it was somehow working, which I suspect is using the older cache version on my Linux server).
  4. Somehow, I just checked it, and it seems to work now. So was the problem fixed now? That would be great! THx.
thiagocrepaldi commented 3 years ago

Hi @cwchung85 I am glad things are working now, but I don't believe there was any action from Microsoft's side for this particular issue. CNTK wheel packages are manually tested before being uploaded to pypi.org, so this issue would have been noticed at most after a few days of the release

What I have experienced in the past was backups being rolled back on pypi.org repos, which causes this kind of temporary issues.

If you see this again, let us know!

cwchung85 commented 3 years ago

@thiagocrepaldi , Sorry, I might have jumped the gun to say it was fixed - the 'fix' was only on a tiny conda env running manually on a build server. I didn;t have the result of the CICD pipeline yet, and turns out the CICD pipeline was still broken (as the CICD full build of an image took more an hour or two to finish). I have turning on --no-cache-dir and -v option and still failed with the same error msg (different versions in metadata).

So what exactly is '2.7 vs 2.7.post1' ? I can see that there are two similarly name: cntk-2.7-cp36-cp36m-manylinux1_x86_64.whl#sha256=9873ecfb... cntk-2.7.post1-cp36-cp36m-manylinux1_x86_64.whl#sha256=97cf434783...

  1. What exactly is post1? (this one is new for me)
  2. Which one is supposed to be a good one?
  3. How does pip handle package name with vs without post1? Thanks!
BharathJaina commented 3 years ago

@thiagocrepaldi ,

I'm facing the same issue. We are using cntk as part of our docker build. It worked on our last build(10+ days ago). Today, when we tried to build the docker it failed with the same error.

As mentioned by cwchung85, we cant rely on manual interventions or workarounds.

thiagocrepaldi commented 3 years ago

I will check whether anything was updated - it shouldn't have

thiagocrepaldi commented 3 years ago

I've just uploaded 2.7.post2 that should fix it. Please try again!

WARNING: Keyring is skipped due to an exception: Failed to unlock the collection!
Collecting cntk-gpu
  Downloading cntk_gpu-2.7.post2-cp36-cp36m-manylinux1_x86_64.whl (522.7 MB)
     |████████████████████████████████| 522.7 MB 45 kB/s 
Requirement already satisfied: scipy>=0.17 in ./miniconda3/envs/cntk_py36/lib/python3.6/site-packages (from cntk-gpu) (1.5.4)
Requirement already satisfied: numpy>=1.11 in ./miniconda3/envs/cntk_py36/lib/python3.6/site-packages (from cntk-gpu) (1.19.4)
Requirement already satisfied: numpy>=1.11 in ./miniconda3/envs/cntk_py36/lib/python3.6/site-packages (from cntk-gpu) (1.19.4)
Installing collected packages: cntk-gpu
Successfully installed cntk-gpu-2.7
thiagocrepaldi commented 3 years ago

LMK if that doesnt work for you