package_index.json file signature verification failure for some users

per1234 commented 5 years ago

We have had four reports of this error on the forum in the last two days. The usual trick of clearing out the data folder and trying again didn't work for any of them. One of the users, Cheetor, provided the package_index.json and package_index.json.sig files they get from https://downloads.arduino.cc/packages/package_index.json and https://downloads.arduino.cc/packages/package_index.json.sig:

I compared these to the files I download from the same URLs and found that their package_index.json file was missing the entries for Arduino SAMD Boards 1.8.1 and avrdude 6.3.0-arduino17, but no differences other than that. The checksum of their .sig file matches mine.

Cheetor is in New Zealand and one of the other reporting users (DavidBMason) is as well. The other two haven't provided their location. The problem stopped occurring for DavidBMason before I could get the bad package_index.json and package_index.json.sig files from them: https://forum.arduino.cc/index.php?topic=621637.msg4212584#msg4212584

My hypothesis is that there was a recent update to package_index.json but the new .json file didn't make it to a server that provides the files to people in NZ. However, the new .sig file did make it to that server. So they are getting the old .json file but the new .sig file, thus the signature verification. Further evidence of this is that when Cheetor used TOR with an exit node in the USA they got the new version of package_index.json: https://forum.arduino.cc/index.php?topic=621811.msg4212512#msg4212512

It would be nice if there was some way to make sure that the .json and .sig files will always hit the servers at the same time. I suspect this delay of days on the .json file is a rare glitch but if we regularly have a delay of even minutes that still is going to cause problems for people, more so because of https://github.com/arduino/Arduino/issues/8936.

Forum threads:

pdo-smith commented 5 years ago

file signature verification failed. File ignored

Sadly the problem still exists for me this morning. I see no progress. My Arduino Due sits helplessly on my desk, begging for some code!

Is it not possible to implement some manual work around in the mean time?

bassai-nx01 commented 5 years ago

here is my data:

package_index.json.info.txt package_index.json.sig.info.txt package_index.json.sig.txt package_index.json.txt

red-scorp commented 5 years ago

@rsora Now it does not work again..

package_index.json.sig.txt package_index.json.txt header.json.sig.txt header.json.txt

What also come to my mind is, I have third party packages installed. Can This be a reason for this behavior in some way?

hm-heli commented 5 years ago

@red-scorp: I don't have any third party software installed.

@pdo-smith: I've commented this 2 days ago: I'm using 1.8.5 because my Nanos are not uploaded with newer verisons. When I quit the IDE, package_index.json and package_index.json.sig are deleted from Arduino15. When I copy these two files back from a backup I made earlier, all works fine again, 'till I quit the IDE. Therefore I face actually 2 Problems (in my case): The IDE deletes the jsons and when restarted can't reload them from internet with above mentioned error msg.

chrisoutdoorwork commented 5 years ago

Previously i tried through two different networks: one through a PFSense router that i manage, and the other through the work domain managed network. This morning my laptop did a windows update before i could test the Arduino software again I am now able to do the update and gain access to the correct board again. (arduino uno wifi rev2). This was done through the PFSense router network

package_index.json.sig.txt package_index.json.sig_headers.txt package_index.json.txt package_index.json_headers.txt

TheTrueForce commented 5 years ago

It no longer errors for me.

pdo-smith commented 5 years ago

Even when I download package_index.json.sig and place it in the .arduino15 folder I still get the error. So a manual workaround does not seem possible for me.

pdo-smith commented 5 years ago

I m happy to report that it now works.

pdo-smith commented 5 years ago

And now it fails again, exactly as it did before.

Ranothil commented 5 years ago

still failing https://downloads.arduino.cc/packages/package_index.json file signature verification failed. File ignored.

pdo-smith commented 5 years ago

It is still failing. The library manager works but not the board manager. When I invoke the board manager I get the signature verification failed message.

Serendipia92 commented 5 years ago

Hello everyone,

I am from Spain and I´m having the same issue, I have tried all the solutions posted here, but the problem stills and I don´t know what to do for solving this.

Thank you for the support, and hope this will be fix soon.

rsora commented 5 years ago

Hi, we are very sorry that what we have done until now server side had almost no effect on your issue, we are working both on a Java IDE PR and on our servers in order to resolve these issues, I'll keep you posted on this.

In the meantime, if the Java IDE still fails, please follow the troubleshooting guide (https://github.com/arduino/Arduino/issues/8988#issuecomment-506688855) and provide both files and headers as described in step E if nothing in the previous steps worked.

Providing files and headers helps us a lot in our troubleshooting

Thanks for your time in reporting your issue here!

cfulton commented 5 years ago

It appears to have resolved. Just to be sure I removed the local appdata and I was able to install the latest version for the Arduino Due.

Thanks for the work.

Ranothil commented 5 years ago

package_index_troubleshooting.zip

Issue still ongoing with the board manager. Location: Ohio, USA

jmarquezchinchu commented 5 years ago

Issue still in Spain with Macbook Pro 2018 Mojave 10.14.5 (18F203) and new and older versions of arduino IDE. System restored from one week old Time machine backup and still crash.... Tried with home internet connection and mobile phone WiFi hostpot

Ranothil commented 5 years ago

The following did not work for me: system restore, tethering phone wifi, vpn, or portable version

sharpcodepro commented 5 years ago

Still failing Spain.

Previously i tried through two different networks: one through a PFSense router that i manage, and the other through the work domain managed network. This morning my laptop did a windows update before i could test the Arduino software again I am now able to do the update and gain access to the correct board again. (arduino uno wifi rev2). This was done through the PFSense router network

package_index.json.sig.txt package_index.json.sig_headers.txt package_index.json.txt package_index.json_headers.txt

These files worked for me! Thank you. Those that are live still fail at least for south of Spain.

jmarquezchinchu commented 5 years ago

Still failing Spain.

Previously i tried through two different networks: one through a PFSense router that i manage, and the other through the work domain managed network. This morning my laptop did a windows update before i could test the Arduino software again I am now able to do the update and gain access to the correct board again. (arduino uno wifi rev2). This was done through the PFSense router network package_index.json.sig.txt package_index.json.sig_headers.txt package_index.json.txt package_index.json_headers.txt

These files worked for me! Thank you. Those that are live still fail at least for south of Spain.

Hola, como los has instalado? en Mac o Windows? gracias

pdo-smith commented 5 years ago

This is so strange. Every now and then Board Manager will work properly, giving me the full list of available boards. Then when I retry it, it fails with the now familiar error message. Thereafter it keeps giving me the error message.

sl1pkn07 commented 5 years ago

how to disable signature check?

ghost commented 5 years ago

Still failing in Spain and need it to work package_index.json.sig.txt package_index.json.txt header_json_sig.txt header_json.txt

sharpcodepro commented 5 years ago

Guys it's really easy to fix at least as a temporary solution. Works 100%. I described it here https://sharpcode.pro/fixing-package-index-json-file-signature-verification-problem-in-arduino

Make sure to download this file https://github.com/arduino/Arduino/files/3343499/package_index.json.txt

Hope this helps.

sl1pkn07 commented 5 years ago

@sharpcodepro not works for me

i put your json file in my webserver (https://sl1pkn07.wtf/packages/package_index.json) and paste the URL in the preferences, and not works

https://downloads.arduino.cc/packages/package_index.json la verificación de la firma de archivo falló. Archivo ignorado.
 package_index.json la verificación de la firma de archivo falló
java.lang.RuntimeException: cc.arduino.contributions.SignatureVerificationFailedException: package_index.json la verificación de la firma de archivo falló
    at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:150)
    at java.base/java.lang.Thread.run(Thread.java:835)
Caused by: cc.arduino.contributions.SignatureVerificationFailedException: package_index.json la verificación de la firma de archivo falló
    at cc.arduino.contributions.packages.ContributionsIndexer.parseIndex(ContributionsIndexer.java:91)
    at processing.app.BaseNoGui.initPackages(BaseNoGui.java:484)
    at processing.app.Base$9.onIndexesUpdated(Base.java:1381)
    at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:148)
    ... 1 more

on archlinux Arduino IDE 1.8.9

EDIT: my webserver redirect to https, i've created a simple python webserver (https://daanlenaerts.com/blog/2015/06/03/create-a-simple-http-server-with-python-3/) for http, and still not works

also i tersted this files https://github.com/arduino/Arduino/issues/8988#issuecomment-507078474, the json and the sig file, also not works

pdo-smith commented 5 years ago

@sharpcodepro, Many thanks for your very helpful tip. It solves my problem completely, allowing me to continue using my Due. There is no need for a web server. I used the following line(on Ubuntu):

file:///home/peter/Downloads/arduino15/package_index.json

This still gives the 'file signature verification failed' error message because the Arduino IDE first tries to get the file from its official repository, fails, then moves on to get it successfully from my specified file location.

nachosmooth commented 5 years ago

It's not working in Spain, in two computers with Arduino IDE 1.8.9 for Linux x64

jmarquezchinchu commented 5 years ago

It works with a VPN connecting to Canada, Sweeden, Australia.... For temporal work, works fine

nachosmooth commented 5 years ago

package_index.json.headers.txt package_index.json.txt

atkarim commented 5 years ago

Same for me in Spain, but it's working if I connect to a French network

ghost commented 5 years ago

Suddenly it's working for me in Spain

nachosmooth commented 5 years ago

Now It's working... I will check it later in more computers

gvarisco commented 5 years ago

We have applied a fix 2 hours ago and will be monitoring the situation for the next hours/days. It'd be great if you all could let us know your feedback.

jmarquezchinchu commented 5 years ago

the IDE shows boards but I can't compile with the selected board (mkrwan)

per1234 commented 5 years ago

@jmarquezchinchu that's likely unrelated to this issue. Please post the full error output on the Arduino Forum and we'll help you out over there. If we end up determining your error is caused by a bug in Arduino's software then we can file a bug report here.

pdo-smith commented 5 years ago

I have tested repeatedly and so far it works, perfectly

For interest, what was the fix?

rsora commented 5 years ago

Hi there, As I mentioned in my previous comment we worked on both Server and Java IDE side:

Server Side: We forced the injection of a Cache-Control: private header on our package_index.json and related .sig and .gz file. This should prevent unwanted caching between our CDN and your PCs.

Java IDE Side: We prepared a PR (https://github.com/arduino/Arduino/pull/9023) for the next release that should mitigate the issue enabling caching for the indexes files, without removing them if something in the signature verification process fails. In addition we implemented a simple logging mechanism that should avoid you to search and download the request headers. Sharing the generated logging files will be sufficient.

So,

if you are still experiencing issues related to the indexes files and nothing in the guide posted here (https://github.com/arduino/Arduino/issues/8988#issuecomment-506688855) works
if you want to try a faster preview for the Java IDE
if you want to contribute to the Arduino community improvement

you can download the following preview binaries: https://github.com/arduino/Arduino/pull/9023#issuecomment-508792575, install them, and tell us if it is working for you and solves the signature problem. In case you experience errors using this IDE preview, please do the following steps in order to share with us the IDE logs:

(In the Arduino IDE) File > Preferences
Click the link at the line following "More preferences can be edited directly in the file". This will open the Arduino15 (or similar name depending on OS) folder.
create a zip for the logs folder and upload it as a github comment attachment

this way will be able to see better what gone wrong on your system. Thanks a lot to you all for helping us in solving this (nasty :smile: ) issue!

edit : updated binary download link after some bug fixing made in the PR

sl1pkn07 commented 5 years ago

fixed for me (spain)

red-scorp commented 5 years ago

In my case (Germany) it is very strange. Sometimes it works and I can update/install boards, and few minutes later it does not work any more.

rsora commented 5 years ago

Hi @red-scorp, could you please try using this build for the Java IDE: https://github.com/arduino/Arduino/pull/9023#issuecomment-508792575 and see if it works for you?

As I explained here https://github.com/arduino/Arduino/issues/8988#issuecomment-508127798 we should be able to see what is happening your side inspecting the IDE logs.

Thanks a lot!

edit: Updated link to binaries

pdo-smith commented 5 years ago

After another day of testing it still works for me. Many thanks you guys!

red-scorp commented 5 years ago

@rsora I've downloaded http://downloads.arduino.cc/javaide/pull_requests/arduino-PR-9023-BUILD-874-windows.zip and put it in portable mode. When I start arduino.exe it shows me a splash screen and does not start an editor. Either this version is bad or my local antivirus is evil (which can be). grafik

I'll test on another pc later and let you know..

red-scorp commented 5 years ago

report from another PC: Avast was not happy to run your nonsigned code. Even with disabled antivirus this version is still not able to start editor window.

rsora commented 5 years ago

Hi there, @red-scorp good catch! we fixed the issue you foudt and prepared this new build, I hope that you have the chance to give it a try and let us now how it goes!

To download the updated build use this link: https://github.com/arduino/Arduino/pull/9023#issuecomment-508792575

Thanks a lot!

chrisly42 commented 5 years ago

After hours of unsuccessful attempts, changing the run.options to

run.options=-Djava.net.preferIPv4Stack

in preferences.txt fixed the issue for me.

red-scorp commented 5 years ago

@chrisly42 same for me, but only for a single run. Now the same error appears again.

red-scorp commented 5 years ago

@rsora Now I've tried several times with 1.8.9. I works fine. I can update board packages and stuff.

red-scorp commented 5 years ago

@rsora still working... Was it a server issue?

red-scorp commented 5 years ago

@rsora still working... Tried with 1.8.7 and portable 1.8.9. Both are working good.

sonisonjames commented 5 years ago

Getting the problem in Singapore, error is : package_index.json file signature verification failed java.lang.RuntimeException: cc.arduino.contributions.SignatureVerificationFailedException: package_index.json file signature verification failed at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:150) at java.lang.Thread.run(Thread.java:748) Caused by: cc.arduino.contributions.SignatureVerificationFailedException: package_index.json file signature verification failed at cc.arduino.contributions.packages.ContributionsIndexer.parseIndex(ContributionsIndexer.java:91) at processing.app.BaseNoGui.initPackages(BaseNoGui.java:484) at processing.app.Base$9.onIndexesUpdated(Base.java:1381) at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:148) ... 1 more

Sxyther commented 5 years ago

Got the same issue TODAY!

But I got latest Windows 10 update yesterday..

java version "1.8.0_221" Java(TM) SE Runtime Environment (build 1.8.0_221-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)

and Arduino 1.8.9

Got Java errors when I tried to download code to a SAMD21 MCU over CDC /USB as well and was working fine yesterday... hope this help

package_index.json file signature verification failed java.lang.RuntimeException: cc.arduino.contributions.SignatureVerificationFailedException: package_index.json file signature verification failed at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:150) at java.lang.Thread.run(Thread.java:748) Caused by: cc.arduino.contributions.SignatureVerificationFailedException: package_index.json file signature verification failed at cc.arduino.contributions.packages.ContributionsIndexer.parseIndex(ContributionsIndexer.java:91) at processing.app.BaseNoGui.initPackages(BaseNoGui.java:484) at processing.app.Base$9.onIndexesUpdated(Base.java:1381) at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:148) ... 1 more

arduino / Arduino

package_index.json file signature verification failure for some users #8988