search

piwheels / packages

Issue tracker for piwheels package issues
https://github.com/piwheels/packages/issues
20 stars 5 forks source link

FIXED: Hash sum mismatch sha256 #32

Closed bennuttall closed 5 years ago

bennuttall commented 5 years ago

Have received several reports of hash mismatches when downloading large files.

Update: I believe this is now fixed.

If you still experience this issue, please comment here and use the following workaround:

The temporary solution is to use wget to download the file you are failing to install, e.g:

Downloading https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl (87.1MB)
wget https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl
sudo pip3 install tensorflow-1.13.1-cp37-none-linux_armv6l.whl
lynk1952 commented 5 years ago

I am getting the hash sum mismatch. Rasbianlite (stretch) loaded and updated and upgraded today. Raspberrypi 3 B+ following instructions from http://docs.donkeycar.com/guide/robot_sbc/setup_raspberry_pi/#step-8-install-dependencies running these steps python3 -m virtualenv -p python3 env --system-site-packages echo "source env/bin/activate" >> ~/.bashrc source ~/.bashrc mkdir projects cd projects git clone https://github.com/autorope/donkeycar cd donkeycar git checkout master pip install -e .[pi] pip install tensorflow==1.13.1

everything goes fine until the last step (this is running in a (env)) This is what I get:

Collecting tensorflow==1.13.1 Downloading https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp35-none-linux_armv7l.whl (92.5MB) |██▉ | 8.2MB 1.3MB/s eta 0:01:07 ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them. tensorflow==1.13.1 from https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp35-none-linux_armv7l.whl#sha256=6c00dd13db0791e83cb08d532f007cc7fd44c8d7b52662a4a0065ac4fe7ca18a: Expected sha256 6c00dd13db0791e83cb08d532f007cc7fd44c8d7b52662a4a0065ac4fe7ca18a Got 2adb4f15706a6cf0db4629df647284f9e26bc0640f2599d774c3679c25640525

I tried following the suggestions in #170 but get a not supported on this platform error. When downloading the file I get a lot of retires before it succeeds in downloading.

Any ideas?

lynk1952 commented 5 years ago

After trying to get this to work this afternoon, it appears the problem is that the down load begins and then slows down and whatever process is called to run this times out and quits downloading and creates a hash and it doesn't match. It wouldn't since it didn't complete the download. I have had it fail all over the place between 8mb and 33 mb. I changed from wifi to wired ethernet and get the same results.

pelrun commented 5 years ago

The server is definitely bogging down for some reason before finally throwing an error. Wget is reporting the following each time it needs to retry:

--2019-07-18 16:58:40--  https://www.piwheels.org/simple/home-assistant-frontend/home_assistant_frontend-20190626.0-py3-none-any.whl
Resolving www.piwheels.org (www.piwheels.org)... 46.235.225.189, 93.93.129.174, 2a00:1098:0:80:1000:3b:1:1, ...
Connecting to www.piwheels.org (www.piwheels.org)|46.235.225.189|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 25719197 (25M)
Saving to: ‘home_assistant_frontend-20190626.0-py3-none-any.whl’

home_assistant_frontend-20190626.0-py3-none-any.whl    21%[========================>                                                                                                 ]   5.17M  40.7KB/s    in 2m 35s

2019-07-18 17:01:16 (34.3 KB/s) - Read error at byte 5423104/25719197 (A TLS packet with unexpected length was received.). Retrying.
up2young commented 5 years ago

I failed to install pandas and matplotlib with the same error.But when I wget the files to another sever,I can install them.

ccatterina commented 5 years ago

Hi @bennuttall I noticed that the hash of future package is changed again the last weekend. Fortunately I have saved a copy of the old future wheel, so I could compare the new package with the old one, here's the result:

diff -bur old/future-0.17.1.dist-info/RECORD new/future-0.17.1.dist-info/RECORD
--- old/future-0.17.1.dist-info/RECORD  2019-06-22 23:53:46.000000000 +0200
+++ new/future-0.17.1.dist-info/RECORD  2019-07-18 18:04:50.000000000 +0200
@@ -209,7 +209,7 @@
 past/utils/__init__.py,sha256=615AAEnnuC4NMohv07vqlaEgKZVgGFizk5sznounQ9Y,2633
 future-0.17.1.dist-info/LICENSE.txt,sha256=4cyh70QH2LtuPyLWyO5mDJEMIw_p02Vwx4cQvbwLZEA,1083
 future-0.17.1.dist-info/METADATA,sha256=RSfWf4RDG523_3mBrjh9i13AYLFixMIAcYhEV6_cW3E,3679
-future-0.17.1.dist-info/WHEEL,sha256=_NOXIqFgOaYmlm9RJLPQZ13BJuEIrp5jx5ptRD5uh3Y,92
+future-0.17.1.dist-info/WHEEL,sha256=S8S5VL-stOTSZDYxHyf0KP7eds0J72qrK0Evu3TfyAY,92
 future-0.17.1.dist-info/entry_points.txt,sha256=-ATQtLUC2gkzrCYqc1Twac093xrI164NuMwsRALJWnM,89
 future-0.17.1.dist-info/top_level.txt,sha256=DT0C3az2gb-uJaj-fs0h4WwHYlJVDp0EvLdud1y5Zyw,38
 future-0.17.1.dist-info/RECORD,,
diff -bur old/future-0.17.1.dist-info/WHEEL new/future-0.17.1.dist-info/WHEEL
--- old/future-0.17.1.dist-info/WHEEL   2019-06-22 23:53:44.000000000 +0200
+++ new/future-0.17.1.dist-info/WHEEL   2019-07-18 18:04:50.000000000 +0200
@@ -1,5 +1,5 @@
 Wheel-Version: 1.0
-Generator: bdist_wheel (0.32.3)
+Generator: bdist_wheel (0.33.4)
 Root-Is-Purelib: true
 Tag: py3-none-any

I attach the packages: whl.zip

bennuttall commented 5 years ago

Ah, so that I can explain. While we were not building I manually built and imported the top most downloaded packages. Once we completed new builds I deleted the builds and let them be built again the normal way. All it takes for wheels to differ is for them to have been built at a different time (as your diff shows).

bfabio commented 5 years ago

All it takes for wheels to differ is for them to have been built at a different time (as your diff shows).

Thanks, that explains it! The different digest actually stems from the different wheel version, which results in a different WHEEL file.

bennuttall commented 5 years ago

Had a few more reports of this, and I've managed to reproduce it myself:

pi@raspberrypi:~$ pip3 wheel tensorflow --no-deps
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting tensorflow
  Downloading https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl (87.1MB)
    23% |███████▋                        | 20.6MB 678kB/s eta 0:01:38
THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    tensorflow from https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl#sha256=25f4ff027beec1e568baf8e90a07bad59d354560533d6b37318b9efeb70beeb1:
        Expected sha256 25f4ff027beec1e568baf8e90a07bad59d354560533d6b37318b9efeb70beeb1
             Got        2bb8824416ef5e287b4e01f8120164de5d38bda996b84d0d2de74038f839932d
pi@raspberrypi:~$ pip3 wheel scipy --no-deps
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting scipy
  Downloading https://www.piwheels.org/simple/scipy/scipy-1.3.0-cp37-cp37m-linux_armv6l.whl (46.6MB)
    47% |███████████████▏                | 22.1MB 646kB/s eta 0:00:38
THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    scipy from https://www.piwheels.org/simple/scipy/scipy-1.3.0-cp37-cp37m-linux_armv6l.whl#sha256=6d739ec72971529d15de4ba433568045e766eb9fe94e3d31afd67957b07630c4:
        Expected sha256 6d739ec72971529d15de4ba433568045e766eb9fe94e3d31afd67957b07630c4
             Got        0caf07efdbc721e86339ee72308ae216fbfb647ae998ce4dd26e1e38b7a53715

What's happening is the download is timing out due to a read error, and pip doesn't retry. However, downloading the file with wget experiences the same issue but manages to retry and continue.

pi@raspberrypi:~$ wget https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl
--2019-08-01 13:55:16--  https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl
Resolving www.piwheels.org (www.piwheels.org)... 46.235.225.189, 93.93.129.174, 2a00:1098:0:80:1000:3b:1:1, ...
Connecting to www.piwheels.org (www.piwheels.org)|46.235.225.189|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 87138952 (83M)
Saving to: ‘tensorflow-1.13.1-cp37-none-linux_armv6l.whl’

tensorflow-1.13.1-cp37-no  51%[================>                  ]  42.48M   299KB/s    in 46s     

2019-08-01 13:56:03 (937 KB/s) - Read error at byte 44548096/87138952 (Error decoding the received TLS packet.). Retrying.

--2019-08-01 13:56:04--  (try: 2)  https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl
Connecting to www.piwheels.org (www.piwheels.org)|46.235.225.189|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 87138952 (83M), 42590856 (41M) remaining
Saving to: ‘tensorflow-1.13.1-cp37-none-linux_armv6l.whl’

tensorflow-1.13.1-cp37-no  75%[+++++++++++++++++========>         ]  62.84M   393KB/s    in 24s     

2019-08-01 13:56:28 (881 KB/s) - Read error at byte 65896448/87138952 (Error decoding the received TLS packet.). Retrying.

--2019-08-01 13:56:30--  (try: 3)  https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl
Connecting to www.piwheels.org (www.piwheels.org)|46.235.225.189|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 87138952 (83M), 21242504 (20M) remaining
Saving to: ‘tensorflow-1.13.1-cp37-none-linux_armv6l.whl’

tensorflow-1.13.1-cp37-no 100%[++++++++++++++++++++++++++========>]  83.10M  1.14MB/s    in 21s     

2019-08-01 13:56:51 (1002 KB/s) - ‘tensorflow-1.13.1-cp37-none-linux_armv6l.whl’ saved [87138952/87138952]

We need to work out if this is an issue with piwheels master, or the network itself.

shadetree01010100 commented 5 years ago

Another one, I've been banging my head on the desk all morning. The above work around, using wget to fetch the wheels, is effective in my case, although extremely tedious.

(env) root@raspberrypi:/home/pi# pip install face_recognition==1.2
Collecting face_recognition==1.2
  Downloading https://www.piwheels.org/simple/face-recognition/face_recognition-1.2.0-py2.py3-none-any.whl
Requirement already satisfied: scipy>=0.17.0 in /opt/nio/env/lib/python3.5/site-packages (from face_recognition==1.2)
Collecting face-recognition-models>=0.3.0 (from face_recognition==1.2)
  Downloading https://www.piwheels.org/simple/face-recognition-models/face_recognition_models-0.3.0-py2.py3-none-any.whl (100.6MB)
    71% |██████████████████████▊         | 71.5MB 5.2MB/s eta 0:00:06
Requirement already satisfied: Click>=6.0 in /opt/nio/env/lib/python3.5/site-packages (from face_recognition==1.2)
Requirement already satisfied: Pillow in /opt/nio/env/lib/python3.5/site-packages (from face_recognition==1.2)
Requirement already satisfied: numpy in /opt/nio/env/lib/python3.5/site-packages (from face_recognition==1.2)
Requirement already satisfied: dlib>=19.7 in /opt/nio/env/lib/python3.5/site-packages (from face_recognition==1.2)
THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    face-recognition-models>=0.3.0 from https://www.piwheels.org/simple/face-recognition-models/face_recognition_models-0.3.0-py2.py3-none-any.whl#sha256=8d6b0af2e37a17120c3f13107974bc252142a4ffcb4e58eabdfcf26608e52c24 (from face_recognition==1.2):
        Expected sha256 8d6b0af2e37a17120c3f13107974bc252142a4ffcb4e58eabdfcf26608e52c24
             Got        00dea804528814d16776cdba9bfd73d4fee717af8b140f2f26309d78403621e6
bennuttall commented 5 years ago

Another one, I've been banging my head on the desk all morning.

We're looking into it. This issue is pinned so that people find the workaround quickly.

The above work around, using wget to fetch the wheels, is effective in my case, although extremely tedious.

You're welcome. Try deleting /etc/pip.conf and seeing how tedious life without piwheels is!

shadetree01010100 commented 5 years ago

@bennuttall I did not mean to be rude, I meant to post a confirmation that the work-around works as described in my case; and in my frustration I seriously under-stated the relief and gratitude that I feel having found this sticky. I actually came here to post a bug report for what I thought might be a broken package... quite frankly I'm disappointed in pip.

Thanks for everything you do!

bennuttall commented 5 years ago

Think this is fixed now - please see update note above

pelrun commented 5 years ago

Just encountered this again. :(

ERROR:homeassistant.util.package:Unable to install package home-assistant-frontend==20190828.0: ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    home-assistant-frontend==20190828.0 from https://www.piwheels.org/simple/home-assistant-frontend/home_assistant_frontend-20190828.0-py3-none-any.whl#sha256=45459517d4c0d41d217c93ad1e457fc7cf589bbeb041d69677359b9d65f61f62 (from -c /srv/homeassistant/lib/python3.7/site-packages/homeassistant/package_constraints.txt (line 14)):
        Expected sha256 45459517d4c0d41d217c93ad1e457fc7cf589bbeb041d69677359b9d65f61f62
             Got        f12ac6a34717916f69d80d0e93a5962b59af7fba0d3c717c242c2e6edd9cfe79
waveform80 commented 5 years ago

@pelrun did the workaround work for you? I just want to make sure it's definitely the same issue in which case I can just bump the Apache timeout up again

pelrun commented 5 years ago

Yup, wget still works, and it took noticeably longer before failing than last time.

waveform80 commented 5 years ago

Yup, wget still works, and it took noticeably longer before failing than last time.

Yeah, that'll be because we bumped the timeout. I've now bumped it to 30 seconds (incidentally this isn't a timeout for the transfer to finish, but for the server to wait for communication from the client, e.g. for acknowledgement of data). I'd be surprised if any pi gets tied up doing other things for more than 30 seconds (and thus is unable to respond with acknowledgements and the like).

That said, I don't think this issue will ever go away entirely, all we can really do is minimize it as much as possible. The underlying issue here is with pip which performs an incomplete transfer and, despite knowing the transfer isn't complete, still tries to verify and install the wheel anyway (?!). I'm guessing there may be some historical reason for this (e.g. it's possible to have arbitrary length HTTP and FTP responses, but obviously that's not the case here otherwise it wouldn't be possible to display the progress of the download). Anyway, when I get a spare moment, I might try and dig into pip's transfer code and see if I can fix it (@bennuttall fancy filing a bug with them so there's a bit of visibility on it?).

Also, just to head off any questions about why our timeout's less than Apache's default (which is 60 seconds): the longer the timeout, the longer failed/idle connections tend to hang around (taking up resources). Given that the master is a pi, we're pretty aggressive with minimizing such things.

hectorm commented 4 years ago

I'm experiencing this problem with tensorflow-1.13.1-cp37-none-linux_armv6l.whl.

THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    tensorflow==1.13.1 from https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl#sha256=25f4ff027beec1e568baf8e90a07bad59d354560533d6b37318b9efeb70beeb1:
        Expected sha256 25f4ff027beec1e568baf8e90a07bad59d354560533d6b37318b9efeb70beeb1
             Got        3aad2c162168a62adae30e226bcddfef74e5db1ca98bec1383958fb580e67123
bennuttall commented 4 years ago

Ah, looks like the armv6 wheel got overwritten by the armv7 wheel. I've re-imported the armv6 wheel and it's now showing the right hash. Can you verify? I think you might need to use --no-cache-dir in your pip command.

hectorm commented 4 years ago

Yes, it's fixed. Thanks!

ve6rah commented 4 years ago

Same issue again:

2020-01-15 14:52:20 ERROR (SyncWorker_4) [homeassistant.util.package] Unable to install package home-assistant-frontend==20200108.0: ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
    home-assistant-frontend==20200108.0 from https://www.piwheels.org/simple/home-assistant-frontend/home_assistant_frontend-20200108.0-py3-none-any.whl#sha256=72875c98160e6317bf4b5e06c7d2bcdcf0a660b1b2ee94d163f038ec64f9b650 (from -c /srv/homeassistant/lib/python3.7/site-packages/homeassistant/package_constraints.txt (line 14)):
        Expected sha256 72875c98160e6317bf4b5e06c7d2bcdcf0a660b1b2ee94d163f038ec64f9b650
             Got        3cea10ae3000fbbd64a05b7bc33fea58c6c02eec545c42b229480df61ee5bc3f

workaround works.

bennuttall commented 4 years ago

The hash is correct on piwheels. I think this is a bug in pip, that fails to complete a download but continues regardless.

hoefling commented 4 years ago

Wheels

tensorflow-1.14.0-cp36-none-linux_armv7l.whl
tensorflow-1.14.0-cp36-none-linux_armv6l.whl
tensorflow-1.14.0-cp35-none-linux_armv7l.whl
tensorflow-1.14.0-cp35-none-linux_armv6l.whl
tensorflow-1.14.0-cp34-none-linux_armv7l.whl
tensorflow-1.14.0-cp34-none-linux_armv6l.whl

all have the same hash in URLs.

bennuttall commented 4 years ago

Yes, they're all copies of the same file. This is unusual but tensorflow's build process means the wheel is compatible with different Python versions.

hoefling commented 4 years ago

Are you sure the hashes are correct? pip hash reports a mismatch, e.g.

pip hash tensorflow-1.14.0-cp34-none-linux_armv7l.whl
tensorflow-1.14.0-cp34-none-linux_armv7l.whl:
--hash=sha256:cba22b6d9a3e7a92c07e142bd5256c9773fd20c18090cb1d222357d3b3028655

is correct, but

$ pip hash tensorflow-1.14.0-cp35-none-linux_armv6l.whl 
tensorflow-1.14.0-cp35-none-linux_armv6l.whl:
--hash=sha256:65c83ef17cd950cf40d021070f3e7e1fa99499a99815c15495920ddc3440a98f
$ pip hash tensorflow-1.14.0-cp34-none-linux_armv6l.whl
tensorflow-1.14.0-cp34-none-linux_armv6l.whl:
--hash=sha256:65c83ef17cd950cf40d021070f3e7e1fa99499a99815c15495920ddc3440a98f

are not (didn't test them all though).

Edit: looks like al linux_armv7l wheels have the correct hash, but linux_armv6l ones not (should be 65c83ef17cd950cf40d021070f3e7e1fa99499a99815c15495920ddc3440a98f)

bennuttall commented 4 years ago

Yep - looks like you're right. I've updated the hashes - can you check?

hoefling commented 4 years ago

Looks good to me now, thank you for the quick response!

noorkhokhar99 commented 9 months ago

upgrade your pip first, then reinstall it solution is available

https://www.youtube.com/@Pyresearch

Screenshot 2023-12-17 at 3 14 55 PM