search

arduino / Arduino

Arduino IDE 1.x
https://www.arduino.cc/en/software
Other
14.11k stars 7k forks source link

package_index.json file signature verification failure for some users #8988

Open per1234 opened 5 years ago

per1234 commented 5 years ago

We have had four reports of this error on the forum in the last two days. The usual trick of clearing out the data folder and trying again didn't work for any of them. One of the users, Cheetor, provided the package_index.json and package_index.json.sig files they get from https://downloads.arduino.cc/packages/package_index.json and https://downloads.arduino.cc/packages/package_index.json.sig:

I compared these to the files I download from the same URLs and found that their package_index.json file was missing the entries for Arduino SAMD Boards 1.8.1 and avrdude 6.3.0-arduino17, but no differences other than that. The checksum of their .sig file matches mine.

Cheetor is in New Zealand and one of the other reporting users (DavidBMason) is as well. The other two haven't provided their location. The problem stopped occurring for DavidBMason before I could get the bad package_index.json and package_index.json.sig files from them: https://forum.arduino.cc/index.php?topic=621637.msg4212584#msg4212584

My hypothesis is that there was a recent update to package_index.json but the new .json file didn't make it to a server that provides the files to people in NZ. However, the new .sig file did make it to that server. So they are getting the old .json file but the new .sig file, thus the signature verification. Further evidence of this is that when Cheetor used TOR with an exit node in the USA they got the new version of package_index.json: https://forum.arduino.cc/index.php?topic=621811.msg4212512#msg4212512

It would be nice if there was some way to make sure that the .json and .sig files will always hit the servers at the same time. I suspect this delay of days on the .json file is a rare glitch but if we regularly have a delay of even minutes that still is going to cause problems for people, more so because of https://github.com/arduino/Arduino/issues/8936.

Forum threads:

facchinm commented 5 years ago

@rsora or @endorama may you take a look?

endorama commented 5 years ago

@rsora will take care on this. I'm subscribed to the issue, so just ping if you need me!

rsora commented 5 years ago

I'm reviewing the issue and opening an internal Incident report, I'll keep you posted!

ilmarhrundel commented 5 years ago

I have the same problem on two PCs and four different internet connections..

hortynz commented 5 years ago

I am in NZ too, and I also had the same problem on two different PCs across several web connections. I didn't try tunnelling yet - would have to set something up. I'm still experiencing the issue this morning, and have attached the two files I get. package_index.json.txt package_index.json.sig.txt

hortynz commented 5 years ago

not sure if searches will scan the text of our issue, but just in case, I'm pasting the error message displayed on my sketch UI. "package_index.json file signature verification failed. File ignored"

per1234 commented 5 years ago

Thanks for sharing those files @hortynz! They have the same issue as the files shared on the forum by Cheetor. package_index.json is missing the entries for Arduino SAMD Boards 1.8.1 and avrdude 6.3.0-arduino17 that are present in my download from https://downloads.arduino.cc/packages/package_index.json but the checksum of the .sig file matches the checksum of the .sig file I get from https://downloads.arduino.cc/packages/package_index.json.sig.

rsora commented 5 years ago

Hi there, Can someone provide the http headers that are received when calling both https://downloads.arduino.cc/packages/package_index.json https://downloads.arduino.cc/packages/package_index.json.sig in case you are still experiencing the signature error? Thanks!

anzas commented 5 years ago

Here are the http headers package_index_json_header.txt package_index_json_sig_header.txt

rsora commented 5 years ago

Hi there, I' have just launched a CDN refresh can you please verify if you are still experiencing the _"packageindex.json file signature verification failed. File ignored" error? If yes please reply with geographical location and both headers and files. Thanks! cc @hortynz @per1234 @anzas

anzas commented 5 years ago

That fixed it, at least for me. Location is Finland. header_json.txt header_json_sig.txt package_index.json.sig.txt package_index.json.txt

ilmarhrundel commented 5 years ago

Everything is working now! (Estonia) Thank You very much.

hortynz commented 5 years ago

here too, thanks

cheetor5923 commented 5 years ago

Can confirm, CDN refresh has worked. I'm now getting the correct file

bhavanakrishna commented 5 years ago

http://arduino.esp8266.com/stable/package_esp8266com_index.json file signature verification failed. File ignored. help me out with this error i tried different versions of arduino ide and different system but same error

rsora commented 5 years ago

Hi @bhavanakrishna, the _package_esp8266comindex.json is a third party index that is not served by our services, I suggest you to ask in the https://www.esp8266.com/ forum for hints on how to solve your problem!

Regarding this issue, having received a positive feedback both in the forum and in the previous message, I'll proceed to close and solve this issue.

Thanks to @hortynz @per1234 @anzas @cheetor5923 @ilmarhrundel for providing feedback!

BZ840 commented 5 years ago

Hi @rsora, I encountered the same issue on multiple devices since yesterday: https://downloads.arduino.cc/packages/package_index.json file signature verification failed. File ignored. Here are the files: package_index.json.txt

Thank you for your help in advance!

endorama commented 5 years ago

Hi @BZ840, please provide the HTTP headers you get when requesting the files, so we can look into it!

BZ840 commented 5 years ago

Hi @endorama, Here is the file http_header.txt

Thanks!

endorama commented 5 years ago

Hi @BZ840 sorry for not being clearer, but we need both .json and .json.sig requests headers to debug this.

As reference take this previous comment.

May you also share which nation are you connecting from?

Thanks

BZ840 commented 5 years ago

Hi @endorama header_json.txt header_json.sig.txt The location is Canada.

ghost commented 5 years ago

I have the same problem and I'm from Canada.

https://downloads.arduino.cc/packages/package_index.json https://downloads.arduino.cc/packages/package_index.json.sig

It's causing all MKR boards to not be listed and I can't search them in my boards manager. Every once in a while they will pop up again and be listed, but most of the time they are not. It's random and annoying. All of my university projects rely on the MKR1000.

red-scorp commented 5 years ago

Same problem OS: Windows Location: Germany OS Language: US

package_index.json.sig.txt package_index.json.txt

How can I grab the http header? How can I launched a CDN refresh? Thanks in advance!

per1234 commented 5 years ago

How can I grab the http header?

  1. Open a new browser tab or window.
  2. Press F12 to open the toolbox.
  3. Click the "Network" tab of the toolbox.
  4. On the next bar down in the toolbox, click "All".
  5. Paste the URL (https://downloads.arduino.cc/packages/package_index.json or https://downloads.arduino.cc/packages/package_index.json.sig) into the URL bar of your browser.
  6. Press Enter.
  7. In the toolbox, click on the line that says "package_index.json" or "package_index.json.sig" (depending on which URL you're currently getting the headers for).
  8. In the pane that appears, click the "Headers" tab.
  9. If using Firefox, switch the "Raw headers" switch to the on position for the "Response headers" section.
  10. Click and drag to select all text in the "Response headers" section.
  11. Press Ctrl + C to copy the selected text to the clipboard.
  12. You can now either paste the copied header text directly into a reply here or save it in a .txt file and attach the .txt file in a reply here.
  13. Repeat the process for the other URL.
BZ840 commented 5 years ago

Hi @endorama Those two header files I provided 3 days ago, were used some http header online service to retrieve by pasting the URL. Here is what I got if I follow what @per1234 suggested: header.json.txt header.json.sig.txt The location is Canada.

I apologize for any confusion and inconvenience.

red-scorp commented 5 years ago

https://downloads.arduino.cc/packages/package_index.json

Cache-Control: public, max-age=2419200
CF-Cache-Status: HIT
CF-Ray: 4ec183b538dd7cd6-MUC
Connection: keep-alive
Date: Mon, 24 Jun 2019 20:40:02 GMT
ETag: "266a3f9e6b360886220fc1815d356bfe"
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Expires: Mon, 22 Jul 2019 20:40:02 GMT
Last-Modified: Fri, 24 May 2019 13:52:41 GMT
Server: cloudflare
Vary: Accept-Encoding
Via: 1.1 acc9aed747aea07d6138203ddfb2dcd9.cloudfront.net (CloudFront)
X-Amz-Cf-Id: 6WmU3rql1m5EWvvQmQtbFw5hnIqpSx8W49T9zwQ-XWqJ6TPNYxL-pw==
x-amz-replication-status: COMPLETED
x-amz-version-id: aujWQVWH2.tNQmcPhdSJiy.4gqfqDncA
X-Cache: RefreshHit from cloudfront
red-scorp commented 5 years ago

https://downloads.arduino.cc/packages/package_index.json.sig

Accept-Ranges: bytes
Cache-Control: public, max-age=2419200
CF-Cache-Status: HIT
CF-Ray: 4ec17f72e94f7cd6-MUC
Content-Length: 543
Content-Type: application/pgp-signature
Date: Mon, 24 Jun 2019 20:37:07 GMT
ETag: "0023b680091faf0064985b79da7020a7"
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Expires: Mon, 22 Jul 2019 20:37:07 GMT
Last-Modified: Wed, 19 Jun 2019 15:10:02 GMT
Server: cloudflare
Vary: Accept-Encoding
Via: 1.1 81f038b63d8af92c2b360530d51919c2.cloudfront.net (CloudFront)
X-Amz-Cf-Id: aeZ3vmVN7ICrQWltZ8eqJmvNkgPl5M2lEdoOCs3YynppQWBJ3vorkw==
X-Amz-Cf-Pop: MUC50-C1
x-amz-replication-status: COMPLETED
x-amz-version-id: g4dBS._ndEMgrPdvRNS9v.vv7ZBbCxJX
X-Cache: RefreshHit from cloudfront
red-scorp commented 5 years ago

As I mentioned earlier, location is Germany. The same happened on another PC in different network. I can provide readers for this PC also if you want.

rsora commented 5 years ago

Thanks for the heads up, I'm re-opening the issue and start the investigations. I'll give you all updates once available.

rsora commented 5 years ago

I tried to replicate your problems using the files and headers you provided but I have inconsistent behaviors:

@BZ840:

@red-scorp

@CLK88

Having these behaviors, I ask you all to follow the guide below in order to help you to both solving your issues, or in the worst case, obtain all the information to assess properly the problem our side.

Step A

  1. (In the Arduino IDE) File > Preferences
  2. Click the link at the line following "More preferences can be edited directly in the file". This will open the Arduino15 (or similar name depending on OS) folder.
  3. Delete all files in that folder except for preferences.txt. Please be very careful when deleting things on your computer. When in doubt, back up!
  4. (In the Arduino IDE) Close the Boards Manager window if it's open.
  5. Tools > Board > Boards Manager
  6. Wait for the downloads to finish.

Does the "file signature verification failed" error still occur?

No It was likely caused by a temporary glitch in the Arduino IDE and should now be fixed.

Yes Go on to the next step:

Step B

  1. Close all Arduino IDE windows.
  2. If using Windows, download the "Windows ZIP file for non admin install" version of the Arduino IDE from the Software page. Unzip the downloaded file to any convenient location on your computer where you have write access. Don't put it under C:\Program Files or C:\Program Files (x86) because Windows 10 places extra restrictions on those folders.
  3. Create a folder named portable in the Arduino IDE installation folder to set the Arduino IDE into portable mode. If you are using macOS, you can follow these instructions.
  4. Start the Arduino IDE.
  5. Tools > Board > Boards Manager
  6. Wait for the downloads to finish.

Does the "file signature verification failed" error still occur?

No It may have been caused by your antivirus software being more restrictive of the location of the Arduino15 folder. You may be able to fix the issue by adjusting your antivirus software's settings.

Yes Go on to the next step:

Step C

  1. TEMPORARILY disable your antivirus software.
  2. (In the Arduino IDE) Close the Boards Manager window if it's open.
  3. Tools > Board > Boards Manager
  4. Wait for the downloads to finish.
  5. Re-enable your antivirus software immediately.

Does the "file signature verification failed" error still occur?

No You'll need to whitelist the appropriate folder, file, or process in your antivirus software's settings.

Yes Go on to the next step:

Step D

  1. Connect to the Internet through a different network. If you're currently using the network at your work, try your home network. You can try connecting via the WiFi hotspot on your phone.
  2. (In the Arduino IDE) Close the Boards Manager window if it's open.
  3. Tools > Board > Boards Manager
  4. Wait for the downloads to finish.

Does the "file signature verification failed" error still occur?

No The issue may be caused by the firewall on the original network.

Yes Go on to the next step:

Step E

  1. Open a new browser tab or window.
  2. Press F12 to open the toolbox.
  3. Click the "Network" tab of the toolbox.
  4. On the next bar down in the toolbox, click "All".
  5. Check the box next to "Disable cache".
  6. Paste the URL (https://downloads.arduino.cc/packages/package_index.json or https://downloads.arduino.cc/packages/package_index.json.sig) into the URL bar of your browser.
  7. Press Enter.
  8. In the toolbox, click on the line that says "package_index.json" or "package_index.json.sig" (depending on which URL you're currently getting the headers for).
  9. In the pane that appears, click the "Headers" tab.
  10. If using Firefox, switch the "Raw headers" switch to the on position for the "Response headers" section.
  11. Click and drag to select all text in the "Response headers" section.
  12. Press Ctrl + C to copy the selected text to the clipboard.
  13. You can now either paste the copied header text directly into a reply here or save it in a .txt file and attach the .txt file in a reply here.
  14. In the browser window where you have the package_index file open, press Ctrl + S
  15. Save the file.
  16. Attach the saved file to your reply.
  17. Repeat the process for the other URL.

Thanks for your time!

red-scorp commented 5 years ago

@rsora Based on what you said, this is a network/server problem, isn't it?

I made steps A and B, it did not help. I can not perform steps C and D because I have no admin permissions on this PC.

What can I do to make Boards Manager work?

Thanks in advance!

BZ840 commented 5 years ago

Hi @rsora, The issue is now resolved, it is caused by the university wifi. Thank you so much!

hm-heli commented 5 years ago

hi, does still not work for me in Germany. I did steps A through D. If you wish I can provide step E Regrads, Hermann

gdeflaux commented 5 years ago

I've had the same issue from South Africa. Connecting to internet via a VPN in France solved it.

TheTrueForce commented 5 years ago

I'm having the same issue in Australia. What made mine all the weirder was that I'd literally just compiled that sketch for the Due, changed it to the Uno, and then couldn't switch back to the Due.

A: The verification failed. No downloads began B: Same as A. Verification failed, no downloads were started. C: Same as A. D is not an option for me. I do not have access to another internet connection. E: The JSON's response header:

HTTP/1.1 200 OK
Date: Thu, 27 Jun 2019 01:47:36 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
CF-Cache-Status: HIT
Cache-Control: public, max-age=2419200
CF-Ray: 4ed3c1013876da46-SYD
Age: 642972
ETag: W/"eaf07cd550c0f00f2d10e944f58a6356"
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Expires: Thu, 25 Jul 2019 01:47:36 GMT
Last-Modified: Wed, 19 Jun 2019 15:10:01 GMT
Vary: Accept-Encoding
Via: 1.1 30f80e470abaf8292272e872df09bc50.cloudfront.net (CloudFront)
X-Amz-Cf-Id: XrEdxqfeX65vZ00G1Hozcqa_oqjRXlzUAWLLW0S_PDGLFoxUkNINqQ==
X-Amz-Cf-Pop: SFO5-C3
x-amz-replication-status: COMPLETED
x-amz-version-id: KMuAw10TOM8GHmMIU6x._J5N4wt59qdM
X-Cache: RefreshHit from cloudfront
Server: cloudflare
Content-Encoding: gzip

SIG's response header:

HTTP/1.1 200 OK
Date: Thu, 27 Jun 2019 01:50:27 GMT
Content-Type: application/pgp-signature
Content-Length: 543
Connection: keep-alive
CF-Cache-Status: HIT
Cache-Control: public, max-age=2419200
CF-Ray: 4ed3c52d6bbfda46-SYD
Accept-Ranges: bytes
Age: 1405578
ETag: "279cf9c5a421b4b9fa57e7824c426788"
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Expires: Thu, 25 Jul 2019 01:50:27 GMT
Last-Modified: Wed, 05 Jun 2019 15:59:23 GMT
Vary: Accept-Encoding
Via: 1.1 42f9f0e9bd0296c3bb45648019b2dce5.cloudfront.net (CloudFront)
X-Amz-Cf-Id: IDv17AwhqQZ6-0ZENHzL_oEDJ9g-uckRfYy2Fz9d4UlBgLHEIwyW8A==
x-amz-replication-status: COMPLETED
x-amz-version-id: mR_9xNlB.01OMvTT8PzaVrHZk18.YA0K
X-Cache: Miss from cloudfront
Server: cloudflare
per1234 commented 5 years ago

Thanks @TheTrueForce!

Please also attach the package_index.json and package_index.json.sig files in a reply here. You'll need to save these files with a .txt extension so that GitHub will allow them to be attached.

TheTrueForce commented 5 years ago

I have done a couple of refreshes since those logs, first checking libraries and then boards. The sig file is not present in my Arduino15 directory. This may be a problem.

library_index.json.txt

red-scorp commented 5 years ago

package_index.json.sig.txt package_index.json.txt header.json.sig.txt header.json.txt

Again from another PC in Germany... Same issue

cfulton commented 5 years ago

Also having the issue from Australia from about 16 hours ago. package_index.json.txt package_index.json.header.txt package_index.json.sig.txt package_index.json.sig.header.txt

chrisoutdoorwork commented 5 years ago

Another issue from Sydney Australia

package_index.json.sig.txt package_index.json.sig_headers.txt package_index.json.txt package_index.json_headers.txt

cfulton commented 5 years ago

Using a VPN works. So I guess thats a solution for anyone in Australia ATM.

hm-heli commented 5 years ago

hi, I'm using 1.8.5 because my Nanos are not uploaded with newer verisons. When I quit the IDE, package_index.json and package_index.json.sig are deleted from Arduino15. When I copy these two files back from a backup I made earlier, all works fine again, 'till I quit the IDE. Therefore I face actually 2 Problems (in my case): The IDE deletes the jsons and when restarted can't reload them from internet with above mentioned error msg. Regards, Hermann

rsora commented 5 years ago

Hi there, I reviewed all the files you all attached (thanks for your time and for providing them), I came to the conclusion that there should be something in between the CDN servers we use and your PCs that is caching not perfectly the package_index.json and signature.

I reduced the caching duration for these files in order to refresh them more frequently from our side and hoping that all the proxies and caches in the path to your machines will respect those refreshes. In addition I triggered a content invalidation that should force an index refresh right now.

Please @hm-heli, @cfulton, @chrisoutdoorwork, @red-scorp, @TheTrueForce, @gdeflaux can you please try again to check the boards, and in case of problems, follow these steps (we updated them to make them more clear) and report what you get? Thanks a lot for your time.

Step A

  1. (In the Arduino IDE) File > Preferences
  2. Click the link at the line following "More preferences can be edited directly in the file". This will open the Arduino15 (or similar name depending on OS) folder.
  3. Delete all files in that folder except for preferences.txt. Please be very careful when deleting things on your computer. When in doubt, back up!
  4. (In the Arduino IDE) Close the Boards Manager window if it's open.
  5. Tools > Board > Boards Manager
  6. Wait for the downloads to finish. Does the "file signature verification failed" error still occur? No It was likely caused by a temporary glitch in the Arduino IDE and should now be fixed. Yes Go on to the next step:

    Step B

  7. Close all Arduino IDE windows.
  8. If using Windows, download the "Windows ZIP file for non admin install" version of the Arduino IDE from the Software page. Unzip the downloaded file to any convenient location on your computer where you have write access. Don't put it under C:\Program Files or C:\Program Files (x86) because Windows 10 places extra restrictions on those folders.
  9. Create a folder named portable in the Arduino IDE installation folder to set the Arduino IDE into portable mode. If you are using macOS, you can follow these instructions.
  10. Start the Arduino IDE.
  11. Tools > Board > Boards Manager
  12. Wait for the downloads to finish. Does the "file signature verification failed" error still occur? No It may have been caused by your antivirus software being more restrictive of the location of the Arduino15 folder. You may be able to fix the issue by adjusting your antivirus software's settings. Yes Go on to the next step:

    Step C

  13. TEMPORARILY disable your antivirus software.
  14. (In the Arduino IDE) Close the Boards Manager window if it's open.
  15. Tools > Board > Boards Manager
  16. Wait for the downloads to finish.
  17. Re-enable your antivirus software immediately. Does the "file signature verification failed" error still occur? No You'll need to whitelist the appropriate folder, file, or process in your antivirus software's settings. Yes Go on to the next step:

    Step D

  18. Connect to the Internet through a different network. If you're currently using the network at your work, try your home network. You can try connecting via the WiFi hotspot on your phone.
  19. (In the Arduino IDE) Close the Boards Manager window if it's open.
  20. Tools > Board > Boards Manager
  21. Wait for the downloads to finish. Does the "file signature verification failed" error still occur? No The issue may be caused by the firewall on the original network. Yes Go on to the next step:

    Step E

  22. Open a new browser tab or window.
  23. Press F12 to open the toolbox.
  24. Click the "Network" tab of the toolbox.
  25. On the next bar down in the toolbox, click "All".
  26. Check the box next to "Disable cache".
  27. Paste the URL https://downloads.arduino.cc/packages/package_index.json into the URL bar of your browser.
  28. Press Enter.
  29. In the toolbox, click on the line that says "package_index.json".
  30. In the pane that appears, click the "Headers" tab.
  31. If using Firefox, switch the "Raw headers" switch to the on position for the "Response headers" section.
  32. Click and drag to select all text in the "Response headers" section.
  33. Press Ctrl + C to copy the selected text to the clipboard.
  34. You can now either paste the copied header text directly into a reply here or save it in a .txt file and attach the .txt file in a reply here.
  35. In the browser window where you have the package_index file open, press Ctrl + S
  36. Click the "Save" button.
  37. Paste the URL https://downloads.arduino.cc/packages/package_index.json.sig into the URL bar of your browser.
  38. Press Enter.
  39. If the browser prompts you whether you want to download the file, click the "OK" button.
  40. In the toolbox, click on the line that says "package_index.json.sig".
  41. If using Firefox, switch the "Raw headers" switch to the on position for the "Response headers" section.
  42. Click and drag to select all text in the "Response headers" section.
  43. Press Ctrl + C to copy the selected text to the clipboard.
  44. You can now either paste the copied header text directly into a reply here or save it in a .txt file and attach the .txt file in a reply here.
  45. In your downloads folder, you will now have two files: package_index.json and package_index.json.sig. Rename these files to package_index.json.txt and package_index.json.sig.txt. This is necessary because GitHub only allows specific file types to be attached.
  46. Attach package_index.json.txt and package_index.json.sig.txt to your reply here.
hm-heli commented 5 years ago

hi, nothing has changed for me. since I don't get the json files after all, I can't provide them (Unless your are interested in my working backup jsons). Regards, Hermann

rsora commented 5 years ago

@hm-heli, can you follow the step E of the guide posted in my previous comment and upload the files you get and the related http headers? Thanks!

hm-heli commented 5 years ago

hi, I hope i have it right... package_index.json.sig.txt package_index.json.txt

per1234 commented 5 years ago

@hm-heli those are the headers, which is great, but we also need you to provide the actual files you download from the URLs:

hm-heli commented 5 years ago

hi, I hope I got it right this time, although I'm a little confused that I can download the files with the browser but not with the IDE :-) Regards, Hermann header_json.txt header_json_sig.txt package_index.json.sig.txt package_index.json.txt

pdo-smith commented 5 years ago

I have the same problem. South Africa, using Ubuntu 19.04 package_index.json.sig.txt package_index.json.txt header.json.txt header-json.sig.txt

red-scorp commented 5 years ago

@rsora It's suddenly works today. I've tried several times after you triggered a content invalidation. At least Arduino IDE sees all the boards now. Thanks!