GPG Signing Tool using Azure Key Vault

analogrelay commented 6 years ago

Title updated recently to reflect the new status. Arcade support signing JARs, but we want to support producing GPG signatures using Azure Key Vault Keys.

ASP.NET Core SignalR is shipping a Java Client in 2.2, which means the signing process needs to be able to process Java artifacts. There are two parts to this:

Signing the JAR files themselves - ESRP already supports this. @natemcmaster and I believe this will work with the existing signing tool. We just need to specify the correct certificate when we invoke it.
Producing Maven package signatures - The Maven Central repository requires that all packages include a GPG signature as a separate .asc file alongside them. Ideally, this is something the signing tool can do. For this we would need the signtool to be able to take a path to a GPG key file, and then use that key file to produce a signature file. Any file should be eligible for GPG signing. However, if the file is also being ESRP-signed, the GPG signature must be generated over the ESRP-signed asset. We've been provided guidance that we can manage the GPG key/signature ourselves as the JAR files are also signed by the ESRP tools.

Maven packages consist of four files:

A .pom file, which is an XML manifest (like a NuSpec). This file needs to be GPG-signed but not ESRP-signed
3 .jar files (the binaries, the sources for debugging, and the JavaDoc documentation). Each file needs to be both GPG- and ESRP-signed

I think the approach here would be to enable a GPG key to be provided for an asset. If one is provided, the asset is signed after ESRP signing (if any is done). Also, ESRP signing would be made optional, so an asset can be signed with just the GPG key (to handle the case of the .pom file that doesn't require ESRP signing). I'm also thinking that the tool should support pulling the signing key out of Azure Key Vault.

I'm currently exploring the existing tool and am going to play around with sketching out a change in a PR, but I wanted to describe the scenario here in an issue before going too far down a path that doesn't align with other plans :).

Timeline: We're shipping the Java Client as part of 2.2. For preview 1 we're planning to manually sign the binaries, but we'd like to look at integrating this into our automation for preview 2.

maririos commented 6 years ago

FYI @tmat

maririos commented 6 years ago

@anurse great! So far nothing seems not compatible with the approach we want to take. Just be aware that we are currently working on refactoring some components of the tool, for example, PR dotnet/arcade#410

natemcmaster commented 6 years ago

I tried to use Microsoft.DotNet.SignTool 1.0.0-beta.18421.2 to sign a .jar file and it skip the file:

warning : Unidentified artifact type: C:\src\aspnet\SignalR\artifacts\build\signalr-0.1.0-alpha1-10333.jar

Is there any reason SignTool can't just flow unrecognized files to the generated MicroBuild invocation?

JohnTortugo commented 6 years ago

Nate we don't have support for .jar file at this momment. At least the, tool doesn't recognize the ".jar" extension. I think that is just another form of .zip file, right? If yes, it might just be a matter of recognizing the extension.

natemcmaster commented 6 years ago

I think technically a .jar is a zip file, but unlike vsix or nupkg, we don't need the contents extracted. ESRP supports submitting the .jar file directly for code signing.

tmat commented 6 years ago

The reason why we unpack vsix and nupkg is to sign their content prior signing the actual package. Assuming no artifact in .jar file needs to be signed we should treat it like a dll and not like a container - that is not unpack it. It should just be passed thru.

analogrelay commented 6 years ago

Yeah, treat .jar like a .dll, it just needs to be sent up to ESRP and signed. It's treated as a single unit, the contents are not signed.

JohnTortugo commented 6 years ago

We (@maririos + @jcagme) chatted with @natemcmaster today and we settled on a plan for this. I'm going to start working on making the SignToolTask recognize and sign .jar files. I created issue dotnet/arcade#509 to track the work.

JohnTortugo commented 5 years ago

Just an update on this. The SignTool task now is able to recognize any file extension (configuration is needed) assign a certificate name for it and sign it.

maririos commented 5 years ago

@anurse Part 1 is complete. For part 2 "Producing Maven package signatures". Do we still need it? are you still planning on working on it?

analogrelay commented 5 years ago

We do still need it. I haven't had time to get too deep in to it. The first road-block I hit is that we'd like to use Azure Key Vault to store the key. I started tinkering with a tool that could use Key Vault keys to generate GPG-compatible signatures (https://github.com/anurse/kpg) but haven't had time to go further on this. Currently we have to manually produce these signatures prior to publishing.

JohnTortugo commented 5 years ago

@anurse is this blocking to shipping preview on 12/4?

analogrelay commented 5 years ago

No, we've worked around it. It's a still a very manual process though.

analogrelay commented 5 years ago

Reviving this. I've built a tool that can use KeyVault keys to generate GPG signatures (http://github.com/anurse/kpg). @natemcmaster mentioned that we also need to generate GPG signatures for Linux installers. If there's interest, we can look at merging this code in to Arcade in some form (possibly in to SignTool, or a separate tool).

markwilkie commented 5 years ago

Thanks @anurse ! Any thoughts @JohnTortugo ?

JohnTortugo commented 5 years ago

I think it would be nice to have this in SignTool ! @anurse Let me know how can I help with this.

natemcmaster commented 5 years ago

I would actually prefer to have this as a separate tool. SignTool currently depends on MSBuild.exe and can only be used on Windows, but GPG signing is a Linux scenario. Unless we also plan to make SignTool run on Linux, I would prefer to have a separate tool. Otherwise, we would have to copy bits from our Linux build machines onto Windows just to run GPG signing.

analogrelay commented 5 years ago

but GPG signing is a Linux scenario

It's not just a Linux scenario, we sign Java packages with it on Windows, but you are right that it would likely be needed on Linux. I can look at a separate tool.

analogrelay commented 5 years ago

Ok, so here's my plan:

[ ] Send a PR to Arcade adding my Global Tool (dotnet tool install) that produces gpg signatures from Azure Key Vault keys
[ ] Send a second PR to Arcade wrapping that tool in an MSBuild task that is part of the Arcade SDK
[ ] Integrate that MSBuild task in to ASP.NET Core's build process (to pilot the proces)
[ ] Consider integrating the MSBuild task into the standard build targets (i.e. auto-sign things in a certain MSBuild Item Group, etc.)

JohnTortugo commented 5 years ago

On Linux do you just GPG sign these files? SignTool is windows dependent because of MSBuild/MicroBuild, if you don't need to use MicroBuild I think we can manage to run it on Linux. @tmat correct me if I'm wrong please.

Chrisboh commented 1 year ago

@dkurepa I believe the work you are currently doing is covering this. Can you confirm?

dkurepa commented 1 year ago

The work we've done on this topic is specific to the dotnet-release repo, but yes, we are producing GPG signatures, using gpg4win, for the SignalR Java packages, which we are then publishing to the Maven Central Repository via ESRP Release. More information on this topic can be seen at https://github.com/dotnet/arcade-services/issues/2355

dotnet / arcade

GPG Signing Tool using Azure Key Vault #412