Closed bennuttall closed 5 years ago
I am getting the hash sum mismatch. Rasbianlite (stretch) loaded and updated and upgraded today. Raspberrypi 3 B+ following instructions from http://docs.donkeycar.com/guide/robot_sbc/setup_raspberry_pi/#step-8-install-dependencies running these steps python3 -m virtualenv -p python3 env --system-site-packages echo "source env/bin/activate" >> ~/.bashrc source ~/.bashrc mkdir projects cd projects git clone https://github.com/autorope/donkeycar cd donkeycar git checkout master pip install -e .[pi] pip install tensorflow==1.13.1
everything goes fine until the last step (this is running in a (env)) This is what I get:
Collecting tensorflow==1.13.1 Downloading https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp35-none-linux_armv7l.whl (92.5MB) |██▉ | 8.2MB 1.3MB/s eta 0:01:07 ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them. tensorflow==1.13.1 from https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp35-none-linux_armv7l.whl#sha256=6c00dd13db0791e83cb08d532f007cc7fd44c8d7b52662a4a0065ac4fe7ca18a: Expected sha256 6c00dd13db0791e83cb08d532f007cc7fd44c8d7b52662a4a0065ac4fe7ca18a Got 2adb4f15706a6cf0db4629df647284f9e26bc0640f2599d774c3679c25640525
I tried following the suggestions in #170 but get a not supported on this platform error. When downloading the file I get a lot of retires before it succeeds in downloading.
Any ideas?
After trying to get this to work this afternoon, it appears the problem is that the down load begins and then slows down and whatever process is called to run this times out and quits downloading and creates a hash and it doesn't match. It wouldn't since it didn't complete the download. I have had it fail all over the place between 8mb and 33 mb. I changed from wifi to wired ethernet and get the same results.
The server is definitely bogging down for some reason before finally throwing an error. Wget is reporting the following each time it needs to retry:
--2019-07-18 16:58:40-- https://www.piwheels.org/simple/home-assistant-frontend/home_assistant_frontend-20190626.0-py3-none-any.whl
Resolving www.piwheels.org (www.piwheels.org)... 46.235.225.189, 93.93.129.174, 2a00:1098:0:80:1000:3b:1:1, ...
Connecting to www.piwheels.org (www.piwheels.org)|46.235.225.189|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 25719197 (25M)
Saving to: ‘home_assistant_frontend-20190626.0-py3-none-any.whl’
home_assistant_frontend-20190626.0-py3-none-any.whl 21%[========================> ] 5.17M 40.7KB/s in 2m 35s
2019-07-18 17:01:16 (34.3 KB/s) - Read error at byte 5423104/25719197 (A TLS packet with unexpected length was received.). Retrying.
I failed to install pandas and matplotlib with the same error.But when I wget the files to another sever,I can install them.
Hi @bennuttall
I noticed that the hash of future
package is changed again the last weekend.
Fortunately I have saved a copy of the old future
wheel, so I could compare the new package with the old one, here's the result:
diff -bur old/future-0.17.1.dist-info/RECORD new/future-0.17.1.dist-info/RECORD
--- old/future-0.17.1.dist-info/RECORD 2019-06-22 23:53:46.000000000 +0200
+++ new/future-0.17.1.dist-info/RECORD 2019-07-18 18:04:50.000000000 +0200
@@ -209,7 +209,7 @@
past/utils/__init__.py,sha256=615AAEnnuC4NMohv07vqlaEgKZVgGFizk5sznounQ9Y,2633
future-0.17.1.dist-info/LICENSE.txt,sha256=4cyh70QH2LtuPyLWyO5mDJEMIw_p02Vwx4cQvbwLZEA,1083
future-0.17.1.dist-info/METADATA,sha256=RSfWf4RDG523_3mBrjh9i13AYLFixMIAcYhEV6_cW3E,3679
-future-0.17.1.dist-info/WHEEL,sha256=_NOXIqFgOaYmlm9RJLPQZ13BJuEIrp5jx5ptRD5uh3Y,92
+future-0.17.1.dist-info/WHEEL,sha256=S8S5VL-stOTSZDYxHyf0KP7eds0J72qrK0Evu3TfyAY,92
future-0.17.1.dist-info/entry_points.txt,sha256=-ATQtLUC2gkzrCYqc1Twac093xrI164NuMwsRALJWnM,89
future-0.17.1.dist-info/top_level.txt,sha256=DT0C3az2gb-uJaj-fs0h4WwHYlJVDp0EvLdud1y5Zyw,38
future-0.17.1.dist-info/RECORD,,
diff -bur old/future-0.17.1.dist-info/WHEEL new/future-0.17.1.dist-info/WHEEL
--- old/future-0.17.1.dist-info/WHEEL 2019-06-22 23:53:44.000000000 +0200
+++ new/future-0.17.1.dist-info/WHEEL 2019-07-18 18:04:50.000000000 +0200
@@ -1,5 +1,5 @@
Wheel-Version: 1.0
-Generator: bdist_wheel (0.32.3)
+Generator: bdist_wheel (0.33.4)
Root-Is-Purelib: true
Tag: py3-none-any
I attach the packages: whl.zip
Ah, so that I can explain. While we were not building I manually built and imported the top most downloaded packages. Once we completed new builds I deleted the builds and let them be built again the normal way. All it takes for wheels to differ is for them to have been built at a different time (as your diff shows).
All it takes for wheels to differ is for them to have been built at a different time (as your diff shows).
Thanks, that explains it! The different digest actually stems from the different wheel version, which results in a different WHEEL file.
Had a few more reports of this, and I've managed to reproduce it myself:
pi@raspberrypi:~$ pip3 wheel tensorflow --no-deps
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting tensorflow
Downloading https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl (87.1MB)
23% |███████▋ | 20.6MB 678kB/s eta 0:01:38
THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
tensorflow from https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl#sha256=25f4ff027beec1e568baf8e90a07bad59d354560533d6b37318b9efeb70beeb1:
Expected sha256 25f4ff027beec1e568baf8e90a07bad59d354560533d6b37318b9efeb70beeb1
Got 2bb8824416ef5e287b4e01f8120164de5d38bda996b84d0d2de74038f839932d
pi@raspberrypi:~$ pip3 wheel scipy --no-deps
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting scipy
Downloading https://www.piwheels.org/simple/scipy/scipy-1.3.0-cp37-cp37m-linux_armv6l.whl (46.6MB)
47% |███████████████▏ | 22.1MB 646kB/s eta 0:00:38
THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
scipy from https://www.piwheels.org/simple/scipy/scipy-1.3.0-cp37-cp37m-linux_armv6l.whl#sha256=6d739ec72971529d15de4ba433568045e766eb9fe94e3d31afd67957b07630c4:
Expected sha256 6d739ec72971529d15de4ba433568045e766eb9fe94e3d31afd67957b07630c4
Got 0caf07efdbc721e86339ee72308ae216fbfb647ae998ce4dd26e1e38b7a53715
What's happening is the download is timing out due to a read error, and pip doesn't retry. However, downloading the file with wget experiences the same issue but manages to retry and continue.
pi@raspberrypi:~$ wget https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl
--2019-08-01 13:55:16-- https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl
Resolving www.piwheels.org (www.piwheels.org)... 46.235.225.189, 93.93.129.174, 2a00:1098:0:80:1000:3b:1:1, ...
Connecting to www.piwheels.org (www.piwheels.org)|46.235.225.189|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 87138952 (83M)
Saving to: ‘tensorflow-1.13.1-cp37-none-linux_armv6l.whl’
tensorflow-1.13.1-cp37-no 51%[================> ] 42.48M 299KB/s in 46s
2019-08-01 13:56:03 (937 KB/s) - Read error at byte 44548096/87138952 (Error decoding the received TLS packet.). Retrying.
--2019-08-01 13:56:04-- (try: 2) https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl
Connecting to www.piwheels.org (www.piwheels.org)|46.235.225.189|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 87138952 (83M), 42590856 (41M) remaining
Saving to: ‘tensorflow-1.13.1-cp37-none-linux_armv6l.whl’
tensorflow-1.13.1-cp37-no 75%[+++++++++++++++++========> ] 62.84M 393KB/s in 24s
2019-08-01 13:56:28 (881 KB/s) - Read error at byte 65896448/87138952 (Error decoding the received TLS packet.). Retrying.
--2019-08-01 13:56:30-- (try: 3) https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl
Connecting to www.piwheels.org (www.piwheels.org)|46.235.225.189|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 87138952 (83M), 21242504 (20M) remaining
Saving to: ‘tensorflow-1.13.1-cp37-none-linux_armv6l.whl’
tensorflow-1.13.1-cp37-no 100%[++++++++++++++++++++++++++========>] 83.10M 1.14MB/s in 21s
2019-08-01 13:56:51 (1002 KB/s) - ‘tensorflow-1.13.1-cp37-none-linux_armv6l.whl’ saved [87138952/87138952]
We need to work out if this is an issue with piwheels master, or the network itself.
Another one, I've been banging my head on the desk all morning. The above work around, using wget
to fetch the wheels, is effective in my case, although extremely tedious.
(env) root@raspberrypi:/home/pi# pip install face_recognition==1.2
Collecting face_recognition==1.2
Downloading https://www.piwheels.org/simple/face-recognition/face_recognition-1.2.0-py2.py3-none-any.whl
Requirement already satisfied: scipy>=0.17.0 in /opt/nio/env/lib/python3.5/site-packages (from face_recognition==1.2)
Collecting face-recognition-models>=0.3.0 (from face_recognition==1.2)
Downloading https://www.piwheels.org/simple/face-recognition-models/face_recognition_models-0.3.0-py2.py3-none-any.whl (100.6MB)
71% |██████████████████████▊ | 71.5MB 5.2MB/s eta 0:00:06
Requirement already satisfied: Click>=6.0 in /opt/nio/env/lib/python3.5/site-packages (from face_recognition==1.2)
Requirement already satisfied: Pillow in /opt/nio/env/lib/python3.5/site-packages (from face_recognition==1.2)
Requirement already satisfied: numpy in /opt/nio/env/lib/python3.5/site-packages (from face_recognition==1.2)
Requirement already satisfied: dlib>=19.7 in /opt/nio/env/lib/python3.5/site-packages (from face_recognition==1.2)
THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
face-recognition-models>=0.3.0 from https://www.piwheels.org/simple/face-recognition-models/face_recognition_models-0.3.0-py2.py3-none-any.whl#sha256=8d6b0af2e37a17120c3f13107974bc252142a4ffcb4e58eabdfcf26608e52c24 (from face_recognition==1.2):
Expected sha256 8d6b0af2e37a17120c3f13107974bc252142a4ffcb4e58eabdfcf26608e52c24
Got 00dea804528814d16776cdba9bfd73d4fee717af8b140f2f26309d78403621e6
Another one, I've been banging my head on the desk all morning.
We're looking into it. This issue is pinned so that people find the workaround quickly.
The above work around, using wget to fetch the wheels, is effective in my case, although extremely tedious.
You're welcome. Try deleting /etc/pip.conf
and seeing how tedious life without piwheels is!
@bennuttall I did not mean to be rude, I meant to post a confirmation that the work-around works as described in my case; and in my frustration I seriously under-stated the relief and gratitude that I feel having found this sticky. I actually came here to post a bug report for what I thought might be a broken package... quite frankly I'm disappointed in pip.
Thanks for everything you do!
Think this is fixed now - please see update note above
Just encountered this again. :(
ERROR:homeassistant.util.package:Unable to install package home-assistant-frontend==20190828.0: ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
home-assistant-frontend==20190828.0 from https://www.piwheels.org/simple/home-assistant-frontend/home_assistant_frontend-20190828.0-py3-none-any.whl#sha256=45459517d4c0d41d217c93ad1e457fc7cf589bbeb041d69677359b9d65f61f62 (from -c /srv/homeassistant/lib/python3.7/site-packages/homeassistant/package_constraints.txt (line 14)):
Expected sha256 45459517d4c0d41d217c93ad1e457fc7cf589bbeb041d69677359b9d65f61f62
Got f12ac6a34717916f69d80d0e93a5962b59af7fba0d3c717c242c2e6edd9cfe79
@pelrun did the workaround work for you? I just want to make sure it's definitely the same issue in which case I can just bump the Apache timeout up again
Yup, wget still works, and it took noticeably longer before failing than last time.
Yup, wget still works, and it took noticeably longer before failing than last time.
Yeah, that'll be because we bumped the timeout. I've now bumped it to 30 seconds (incidentally this isn't a timeout for the transfer to finish, but for the server to wait for communication from the client, e.g. for acknowledgement of data). I'd be surprised if any pi gets tied up doing other things for more than 30 seconds (and thus is unable to respond with acknowledgements and the like).
That said, I don't think this issue will ever go away entirely, all we can really do is minimize it as much as possible. The underlying issue here is with pip which performs an incomplete transfer and, despite knowing the transfer isn't complete, still tries to verify and install the wheel anyway (?!). I'm guessing there may be some historical reason for this (e.g. it's possible to have arbitrary length HTTP and FTP responses, but obviously that's not the case here otherwise it wouldn't be possible to display the progress of the download). Anyway, when I get a spare moment, I might try and dig into pip's transfer code and see if I can fix it (@bennuttall fancy filing a bug with them so there's a bit of visibility on it?).
Also, just to head off any questions about why our timeout's less than Apache's default (which is 60 seconds): the longer the timeout, the longer failed/idle connections tend to hang around (taking up resources). Given that the master is a pi, we're pretty aggressive with minimizing such things.
I'm experiencing this problem with tensorflow-1.13.1-cp37-none-linux_armv6l.whl
.
THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
tensorflow==1.13.1 from https://www.piwheels.org/simple/tensorflow/tensorflow-1.13.1-cp37-none-linux_armv6l.whl#sha256=25f4ff027beec1e568baf8e90a07bad59d354560533d6b37318b9efeb70beeb1:
Expected sha256 25f4ff027beec1e568baf8e90a07bad59d354560533d6b37318b9efeb70beeb1
Got 3aad2c162168a62adae30e226bcddfef74e5db1ca98bec1383958fb580e67123
Ah, looks like the armv6 wheel got overwritten by the armv7 wheel. I've re-imported the armv6 wheel and it's now showing the right hash. Can you verify? I think you might need to use --no-cache-dir
in your pip command.
Yes, it's fixed. Thanks!
Same issue again:
2020-01-15 14:52:20 ERROR (SyncWorker_4) [homeassistant.util.package] Unable to install package home-assistant-frontend==20200108.0: ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE. If you have updated the package versions, please update the hashes. Otherwise, examine the package contents carefully; someone may have tampered with them.
home-assistant-frontend==20200108.0 from https://www.piwheels.org/simple/home-assistant-frontend/home_assistant_frontend-20200108.0-py3-none-any.whl#sha256=72875c98160e6317bf4b5e06c7d2bcdcf0a660b1b2ee94d163f038ec64f9b650 (from -c /srv/homeassistant/lib/python3.7/site-packages/homeassistant/package_constraints.txt (line 14)):
Expected sha256 72875c98160e6317bf4b5e06c7d2bcdcf0a660b1b2ee94d163f038ec64f9b650
Got 3cea10ae3000fbbd64a05b7bc33fea58c6c02eec545c42b229480df61ee5bc3f
workaround works.
The hash is correct on piwheels. I think this is a bug in pip, that fails to complete a download but continues regardless.
Wheels
tensorflow-1.14.0-cp36-none-linux_armv7l.whl
tensorflow-1.14.0-cp36-none-linux_armv6l.whl
tensorflow-1.14.0-cp35-none-linux_armv7l.whl
tensorflow-1.14.0-cp35-none-linux_armv6l.whl
tensorflow-1.14.0-cp34-none-linux_armv7l.whl
tensorflow-1.14.0-cp34-none-linux_armv6l.whl
all have the same hash in URLs.
Yes, they're all copies of the same file. This is unusual but tensorflow's build process means the wheel is compatible with different Python versions.
Are you sure the hashes are correct? pip hash
reports a mismatch, e.g.
pip hash tensorflow-1.14.0-cp34-none-linux_armv7l.whl
tensorflow-1.14.0-cp34-none-linux_armv7l.whl:
--hash=sha256:cba22b6d9a3e7a92c07e142bd5256c9773fd20c18090cb1d222357d3b3028655
is correct, but
$ pip hash tensorflow-1.14.0-cp35-none-linux_armv6l.whl
tensorflow-1.14.0-cp35-none-linux_armv6l.whl:
--hash=sha256:65c83ef17cd950cf40d021070f3e7e1fa99499a99815c15495920ddc3440a98f
$ pip hash tensorflow-1.14.0-cp34-none-linux_armv6l.whl
tensorflow-1.14.0-cp34-none-linux_armv6l.whl:
--hash=sha256:65c83ef17cd950cf40d021070f3e7e1fa99499a99815c15495920ddc3440a98f
are not (didn't test them all though).
Edit: looks like al linux_armv7l
wheels have the correct hash, but linux_armv6l
ones not (should be 65c83ef17cd950cf40d021070f3e7e1fa99499a99815c15495920ddc3440a98f
)
Yep - looks like you're right. I've updated the hashes - can you check?
Looks good to me now, thank you for the quick response!
upgrade your pip first, then reinstall it solution is available
Have received several reports of hash mismatches when downloading large files.
Update: I believe this is now fixed.
If you still experience this issue, please comment here and use the following workaround:
The temporary solution is to use
wget
to download the file you are failing to install, e.g: