GPG signatures for source validation

NicoHood commented 7 years ago

As we all know, today more than ever before, it is crucial to be able to trust our computing environments. One of the main difficulties that package maintainers of Linux distributions face, is the difficulty to verify the authenticity and the integrity of the source code.

The Arch Linux team would appreciate it if you would provide us GPG signatures in order to verify easily and quickly your source code releases.

Overview of the required tasks:

Create and/or use a 4096-bit RSA keypair for the file signing.
Keep your key secret, use a strong unique passphrase for the key.
Upload the public key to a key server and publish the full fingerprint.
Sign every new git commit and tag.
Create signed compressed (xz --best) release archives
Upload a strong message digest (sha512) of the archive
Configure https for your download server

GPGit is meant to bring GPG to the masses. It is not only a shell script that automates the process of creating new signed git releases with GPG but also comes with this step-by-step readme guide for learning how to use GPG.

Additional Information:

Thanks in advance.

NicoHood commented 7 years ago

@facchinm @cmaglie Can you please sign the new 1.8 release and the arduino-builder release please? It would really help us to provide secure packages of the arduino software and everyone else can securely check their arduino download.

cmaglie commented 7 years ago

We are setting up the keys for the release of 1.8.2

NicoHood commented 7 years ago

So much love <3 (Please use 4k RSA)

cmaglie commented 7 years ago

The git tag is now signed https://github.com/arduino/Arduino/releases/tag/1.8.2

About the signing of the single packages, instead of generating a signature for every package, what about signing the sha512sum.txt? This way we can sign all the packages at once, something like:

https://downloads.arduino.cc/arduino-1.8.2.sha512sum.txt.asc

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

a2273871799ba4437bb8b53488a27cb1881ec266bf12f4ce004cca35637dd9a5ca775cc3f0e7ce82d7c0a03c2733eb00c7f170b634a2053fef1928cd91915737  arduino-1.8.2-linux32.tar.xz
856968ab70519161cac30f4a2db1ff678bcda0c2e056969bdd89046b33e9440548b25ccd0a2abe01a135e5834b7b2debf22cd2c858e24129e18b747bbf3a985c  arduino-1.8.2-linux64.tar.xz
83e63982b1afd32f6df1a63e87cd1d3f59c532719dafe0fd262cfc3fbb96e7208040f89be9211109282305706e6f6518df6e24a1b48e4b62e0e965f9e3f7e96a  arduino-1.8.2-linuxarm.tar.xz
66130d96796c40620bb05806b74ab4961ad3a76eeacbfebc43b45897b1dbf4a405d5f7f62a94730bf8947e3551110c68b9f09612c88922e0090d9625e8c202e2  arduino-1.8.2-macosx.zip
17214275200be2bc25e70f632644ef15289a59939b146b0d63662882e1978e7c59c0f0e3fcd0611686e85baac46c7dbd2767217d1b341c598485b0417fda440d  arduino-1.8.2-windows.exe
65eced00a26761909e49bf569122f98b4f915d08f91af8c167ea54ac6a1ce3bf0dd14426abf4954260beaca4096e129d80e240c698671d5f175ac307f3970417  arduino-1.8.2-windows.zip
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJY0pZwAAoJEJX6b0PiEYjEjiEP/iMDeLQZ41yGas8qmtUlenTw
dxvjCmJFWAqSu0UAcDWSxLSWrGE1kWpCf61d88AX4+azq8AYvJsDwJ3C8UbFnfq2
uzSIegnwC5ZUhCLpgpHrRTCKAiohZyh0Ao/nv9FmB22Ds+w10cKDRJH+ckMil6TD
mOZCq3QgzY0Tun8hhk/Oyd5UxYIsuLYwhPzhhsweGrWr8BVn49gKJeNXLkNb52bb
rmmOaqokgzsqeDxlX2t2FNq74ykWK/VRs2jXCAodSTjWp7VHW1C6yA/VIVSr8CQ+
Bd/1hvNDj+SEenLbyCf6Aj1oTisafXHZSlvBju/Xcyb7kURFJ0J3fZ8jkzQ+8UIv
HB1dR1ctgS0vhrzdzOKzqj0TR+jFS5xrvxBbm2pbtNOe+84DB1GbiWjRSZTa/M3B
flUNfCJiagKUEUSzgpBUinOOckqgruRdzHNhDtsfjtTjPLQ8ibUSJdpTZ1ShhVjq
ikb/3/EEGP62nNTc8K0Se2KQO1tJjMK/Y/dmquVOICc1hhnKUpa+6a97dMhWEqPb
xc5gETR+v8n5N3mSSVHm6O1ociEE90daPSZqeiSZU7WvblOSMgcTx1jTVec5MYY4
JDPd0lIDa8q9/P2YD7RnaeFfQe8YdByZLu/7ByRnecOBVZ/JSxO62Ryg/vQduOIu
0HbWhMIfxDLGufTYetWk
=28JK
-----END PGP SIGNATURE-----

NicoHood commented 7 years ago

The way we build packages is we download the .tar.gz from github and validate this tarfile against the .asc or .sig file. This process is automated in a way that it only trusts the selected GPG keys and that it automatically checks the available signatures.

Signing the shasum file is way more complicated and can cause wrong parsing/security issues. There was some controversial internal discussion about those upstream signatures and they should just be avoided if possible.

What I suggest is to simply publish the release via gpgit thats automates the whole process for you and everything will be signed properly. If you dont want to use that, please sign the github.tar.gz files manually as described in the readme. You should however always verify those before blindly signing them (thats what gpgit also does for you).

The .zip files on downloads.arduino.org should also be signed, as some build deps need to be downloaded from there instead (libastyle for example). This also implies that all subprojects (arduino-builder, ctags etc) needs to be signed too. Best on github and on downloads.arduino.org.

Please also do not use insecure sha1 signatures and upgrade to gpg2 possibly.

cmaglie commented 7 years ago

Signing the shasum file is way more complicated and can cause wrong parsing/security issues

ok fair enough

You should however always verify those before blindly signing them (thats what gpgit also does for you).

The archives are made locally and uploaded to the download server, there is no need to check that the zip archive content, just created, is the same as the source folder being zipped...

Another question: shall we publish the public key on the arduino website?

PS: I'm still confused on how all this infrastructure can improve things (since an attacker that successfully gain access to the webserver can still easily replace all the archives, signatures and public key altogether...)

kevans91 commented 7 years ago

Another question: shall we publish the public key on the arduino website?

Yes, and you should consider posting it on other places as well, such as to a public key server. I use https://pgp.mit.edu/ myself, @NicoHood may have other recommendations.

PS: I'm still confused on how all this infrastructure can improve things (since an attacker that successfully gain access to the webserver can still easily replace all the archives, signatures and public key altogether...)

This is a valid concern, but between submitting your public key to a keyserver and people having this already cached locally, the damage can be minimized. Previously, an attacker could replace everything and no one would be the wiser. Now, an attacker could still replace everything but this could be detected by a careful third party that actually checks signatures and doesn't necessarily rely on the single copy of your public key to be intact and unaltered.

NicoHood commented 7 years ago

The archives are made locally and uploaded to the download server, there is no need to check that the zip archive content, just created, is the same as the source folder being zipped...

Those do not need additional checks, but signatures. I was talking about this github sources here. Those are generated from github and need to be checked before signing. This is where gpgit automates the process.

You need to sign the arduino.cc and the github files as arduino.cc files contain prebuilt releases and github the sources. Some parts of the arduino package are build from source (and more are coming) and a few still rely on the prebuild arduino.cc downloads.

As an alternative you could provide the source files also on arduino.cc and sign them also locally. look at git archive or gpgit (also works without github) for that.

Another question: shall we publish the public key on the arduino website?

@cmaglie Best would be on website, github and keyservers. I prefer hkps://hkps.pool.sks-keyservers.net but they should exchange information. hkps (note the s) stands for the secure upload.

PS: I'm still confused on how all this infrastructure can improve things (since an attacker that successfully gain access to the webserver can still easily replace all the archives, signatures and public key altogether...)

And now it comes into place that your public key is already known by us. On a package update we validate against the known public fingerprint and this will fail if the signature/archive changed.

Thatswhy keeping the gpg key secure with a strong password is of very high importance. As an alternative you can make other trusted users sign your fingerprint to give it more trust. This builds a trust chain. I can for example sign your key, stating that I trust the owner. But thats mostly only used in linux distributions with a master key schema. Thatshy publishing you fingerprint on multiple sources is very important. People wont hopefully hack arduino.cc and github at the same time.

NicoHood commented 7 years ago

The current tar.xz looks better. However inside the files are named wrong due to a wrong use of git archive. A slash at the end is missing to create a folder instead of renaming all files. arduino-1.8.2README.md

I hope this was not produced by my script, otherwise I need to fix it. Edit: seems okay in my script. Check the leading slash: https://github.com/NicoHood/gpgit/blob/master/gpgit.sh#L480

cmaglie commented 7 years ago

Oh yes I created them manually, probably I missed the slash :-/ For now I'll remove them. I'll check better them tomorrow...

NicoHood commented 7 years ago

The new signatures have this fingerprint:

326567C1C6B288DF32CB061A95FA6F43E21188C4 # Arduino Packages <support@arduino.cc>

How was the key generated? Because it is not trusted by default. If it was my script, I need to change that. I am currently searching for the reasons.

The key has no expire date. That is possibly the reason. And beside the issue it is not good to have a key that never expires. Please use a value of 1 or 2 years.

cmaglie commented 7 years ago

I made it, what does it mean that it's not trusted by default?

NicoHood commented 7 years ago

==> Verifying source file signatures with gpg...
    arduino-1.8.2.tar.xz ... FAILED (the public key 326567C1C6B288DF32CB061A95FA6F43E21188C4 is not trusted)
==> ERROR: One or more PGP signatures could not be verified!
==> ERROR: Could not download sources.

I do not know yet why this happens, because it never happened. But I guess its the expire date. Please add an expire date and upload the key again to the servers.

cmaglie commented 7 years ago

Could it be that the key is not marked as trusted in your trust chain?

NicoHood commented 7 years ago

@cmaglie that never happened before, but yes it could be. However its not normal. In any case: Please add an expire date for your own security. I am currently trying to find the cause of this issue.

matthijskooijman commented 7 years ago

@Nicohood, I just downloaded the key and for me it does not look weird:

matthijs@grubby:~$ gpg --recv-keys 326567C1C6B288DF32CB061A95FA6F43E21188C4
gpg: key 95FA6F43E21188C4: public key "Arduino Packages <support@arduino.cc>" imported
gpg: Total number processed: 1
gpg:               imported: 1
matthijs@grubby:~$ gpg --list-key 326567C1C6B288DF32CB061A95FA6F43E21188C4
pub   rsa4096 2017-03-21 [SC]
      326567C1C6B288DF32CB061A95FA6F43E21188C4
uid           [ unknown] Arduino Packages <support@arduino.cc>
sub   rsa4096 2017-03-21 [E]

matthijs@grubby:~$ gpg --edit-key 326567C1C6B288DF32CB061A95FA6F43E21188C4
gpg (GnuPG) 2.1.15; Copyright (C) 2016 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

pub  rsa4096/95FA6F43E21188C4
     created: 2017-03-21  expires: never       usage: SC
     trust: unknown       validity: unknown
sub  rsa4096/57294ACE46740381
     created: 2017-03-21  expires: never       usage: E
[ unknown] (1). Arduino Packages <support@arduino.cc>

This does say "unknown" for trust level, but that's because it has no signatures and thus no signature path to my own key. I'm not sure how that is for other packages, but I suspect they just have a path into the web of trust? In any case, this would be good for the Arduino key as well, to get into the web of trust. Do any Arduino developers have personal keys they can sign it with? I can also sign it with my key if we can do some out-of-band verification of the fingerprint.

cmaglie commented 7 years ago

However its not normal

Well, that is the whole point of the chain of trust: when you download a key from a keyserver you cannot be 100% sure that is genuine and not a fake one, unless the key is signed from a trusted 3rd party. In this case, since the key is not signed by anyone his "trustness" is set to "unknown", that is the lowest level.

NicoHood commented 7 years ago

It was a problem on my side. Sorry. Expire date still makes sense :P

You should add the mentioned (full!) fingerprint to the website along with the signatures and source. It would be nice if you can also sign the other tarfiles from downloads.arduino.cc so every download is signed.

Edit: For example we currently use:

"https://github.com/arduino-libraries/WiFi101-FirmwareUpdater-Plugin/releases/download/v0.9.0/WiFi101-Updater-ArduinoIDE-Plugin-0.9.0.zip"
"https://downloads.arduino.cc/libastylej-2.05.1-3.zip"
"https://downloads.arduino.cc/liblistSerials/liblistSerials-1.4.0.zip"
arduino-ctags source
arduino-builder source

matthijskooijman commented 7 years ago

I just uploaded a signature for the key to the keyservers.

cmaglie commented 7 years ago

Everything should be in place now, I've published the GPG key on the website too: https://www.arduino.cc/en/Main.Software#source

NicoHood commented 7 years ago

@cmaglie Incredible! You signed almost every source. Thanks so much <3

Missing signatures:

And the docs files:

"https://downloads.arduino.cc/reference-1.6.6-3.zip"
"https://downloads.arduino.cc/reference-1.6.6-3.zip.asc"
"https://downloads.arduino.cc/Galileo_help_files-1.6.2.zip"
"https://downloads.arduino.cc/Galileo_help_files-1.6.2.zip.asc"
"https://downloads.arduino.cc/Edison_help_files-1.6.2.zip"
"https://downloads.arduino.cc/Edison_help_files-1.6.2.zip.asc"

NicoHood commented 7 years ago

ctags source is still not signed, avr core also not. The new wifi updater also has no signature. I will postpone updated until all sources are signed https://github.com/arduino-libraries/WiFi101-FirmwareUpdater-Plugin/releases

cmaglie commented 7 years ago

It should signed everything now.

doronbehar commented 6 years ago

Could it be the sam cores are not signed? I get 404 when I try to download https://downloads.arduino.cc/cores/sam-1.6.11.tar.bz2.asc but obviously https://downloads.arduino.cc/cores/sam-1.6.11.tar.bz2 is there.

NicoHood commented 6 years ago

https://github.com/arduino/Arduino/releases/download/1.8.7/arduino-1.8.7.tar.xz Missing. Signature also.

cmaglie commented 6 years ago

Done! with the new script this should not happen anymore in the future (hopefully)...

NicoHood commented 6 years ago

The signature is now available. But the tarfile does not have a separate folder inside. Normally a tarfile contains a folder of the tarfilename where all data is placed. This makes it easier to extract it into its own folder, rather than putting everything into the working directory. Especially for our automated extracting this makes it more complex than needed.

The older releases included such a folder, so I guess this was just accidentally missed. It would be nice if you could fix that :) https://github.com/arduino/Arduino/releases/download/1.8.7/arduino-1.8.7.tar.xz

Also this signature is missing: https://downloads.arduino.cc/libastylej-2.05.1-4.zip.asc

And this one: https://downloads.arduino.cc/liblistSerials/liblistSerials-1.4.1.zip.asc

Signatures are still missing for the avr core too: https://github.com/arduino/ArduinoCore-avr/issues/44

cmaglie commented 6 years ago

ok updated arduino-1.8.7.tar.xz as you suggested (and also the script).

added the other missing signatures

NicoHood commented 6 years ago

Thank you very much!

NicoHood commented 5 years ago

==> ERROR: Failure while downloading https://github.com/arduino-libraries/WiFi101-FirmwareUpdater-Plugin/releases/download/v0.10.4/WiFi101-Updater-ArduinoIDE-Plugin-0.10.4.zip.asc

cmaglie commented 5 years ago

added the signature