Open priteshbandi opened 1 year ago
Thanks @priteshbandi, it's on my list, will share a PoC once it's ready, and we could start from there.
Thanks @patrickzheng200 for proactively working on this. Do you have ECD (Estimated Completion Date) for PoC? Also, can you share any early draft around CLI interface?
Thanks @patrickzheng200 ! There would be some spec work associated for this feature, covering changes (if any) for payload format, signature storage on disk, and trust policy.
@gokarnm @priteshbandi I remember we had some discussions around local artifacts/arbitrary files/signature storage at least one year ago. At that time, we tracked the features in roadmap repo, I did some searches and find related issues like
There were also some discussions and investigations during implementing feature of "sign/verify local OCI artifact", which is currently an experimental feature. But we didn't continue the discussion recently since it is not in v1 scope any more.
This could be a good topic for community meeting, we can discuss it further on the plan. Does it make sense?
I agree that this is a significant feature request for Notation and has been discussed several times. This request was also captured in Requirement (scenario-1-local-build-sign-validate) and Goals.
Before having an ECD for this issue, let's figure out the end-to-end scenarios, how it works with trust policy, have an approachable design proposal and CLI Spec, and split the work items.
Let's discuss it in the next community meeting.
A follow-up question, if signing local arbitrary files is supported, will signing content from OCI Image Layout be replaced accordingly?
Thanks @patrickzheng200 for proactively working on this. Do you have ECD (Estimated Completion Date) for PoC? Also, can you share any early draft around CLI interface?
It's in its initial stage, so I don't have an ECD for now. But as suggested by Yi and Feynman, we could have a discussion on the feature in the next week's meeting. (btw, @priteshbandi @gokarnm this is my current github account: @Two-Hearts, I couldn't get any notification if you ping my previous account. Thanks.)
Here are a few scenarios I am aware of where this will be helpful.
We have teams that produce binaries that are not distributed via OCI registries (think of published on web sites, or package managers, etc.). They would like to produce SBOMs for those binaries and sign the SBOMs with Notation. The resulting artifacts are binary, SBOM, signature of the SBOM (.sig
file for example). Those three artifacts can be published on a web site for download as separate items.
As an expansion of the above scenario. We don't want to change the workflow of the above teams, but we would like to distribute the SBOM via registry. In this scenario the binary, the SBOM and the SBOM signature are created using the above process. However, we would like to use the SBOM and the SBOM signature and publish them to a registry for distribution. What that means is that we would like to take the SBOM file, the SBOM signature file (.sig
file) and pack them as OCI artifacts and push to the registry.
In both scenarios above, we would like to use Notation to verify the signatures.
I assume this will need updates to the specification in addition to the code.
Thanks @toddysm for providing the user scenarios. Here's my design doc: https://hackmd.io/QteHaBQTS-6h-AsU04U1cQ?view
Here's a PoC implementing the above design released on my local branch: https://github.com/Two-Hearts/notation/releases/tag/v1.0.0-arbitraryFile
It's an initial version, and please feel free to leave your comments/suggestions. Thanks. @shizhMSFT @priteshbandi @sajayantony @toddysm @yizha1 @FeynmanZhou @JeyJeyGao
Apart from signing SBOM, I can share two typical scenarios that rely on signing and verifying arbitrary files.
There are some open-source projects that require maintainers to use the GPG key to sign the installation binaries and other release assets when publishing on the release page. Users are recommended to verify the GPG signature with the GPG tool when downloading from the GitHub release page. Just take the ORAS release verification process as an example: https://github.com/oras-project/oras-www/issues/69#issuecomment-1618736015
However, signing and verifying with the GPG key is a bit cumbersome for maintainers and users since GPG has complicated setup steps. Notation could be a good alternative for signing and verifying the installation binary package if signing arbitrary files is supported. Open-source projects like Kubernetes, Argo CD published signed binaries and suggest users verifying them when downloading from the release page.
Some free softwares and organizations also publish their executable binaries and artifacts on FTP servers, central repository (like Nexus), and object storage. This feature is also useful when signing and verifying those binaries in the traditional release process. Project organizations can distribute the signature alongside the binaries to users.
Apart from signing SBOM, I can share two typical scenarios that rely on signing and verifying arbitrary files.
Sign and verify the installation binaries and release assets for GitHub releases
There are some open-source projects that require maintainers to use the GPG key to sign the installation binaries and other release assets when publishing on the release page. Users are recommended to verify the GPG signature with the GPG tool when downloading from the GitHub release page. Just take the ORAS release verification process as an example: oras-project/oras-www#69 (comment)
However, signing and verifying with the GPG key is a bit cumbersome for maintainers and users since GPG has complicated setup steps. Notation could be a good alternative for signing and verifying the installation binary package if signing arbitrary files is supported. Open-source projects like Kubernetes, Argo CD published signed binaries and suggest users verifying them when downloading from the release page.
Sign and verify the installation binaries for traditional software release process
Some free softwares and organizations also publish their executable binaries and artifacts on FTP servers, central repository (like Nexus), and object storage. This feature is also useful when signing and verifying those binaries in the traditional release process. Project organizations can distribute the signature alongside the binaries to users.
Thanks @FeynmanZhou, the above design should cover the scenarios mentioned by you as well.
Thanks @patrickzheng200 ! There would be some spec work associated for this feature, covering changes (if any) for payload format, signature storage on disk, and trust policy.
For the 1st scenario as Toddy described, we need specifications to cover
For the 2nd scenario "Distributing the SBOM via registry", we need spec to cover
We need CLI spec to cover the CLI UX for both scenarios
Thanks @patrickzheng200 ! There would be some spec work associated for this feature, covering changes (if any) for payload format, signature storage on disk, and trust policy.
For the 1st scenario as Toddy described, we need specifications to cover
- signature payload
- storage on disk
- trust policy
- Any changes to the existing signing and verification workflow
For the 2nd scenario "Distributing the SBOM via registry", we need spec to cover
- storage of both SBOM and signature file in registry
- Any changes to the existing verification workflow.
We need CLI spec to cover the CLI UX for both scenarios
@yizha1 Here's the PR for the CLI spec
changes: https://github.com/notaryproject/notation/pull/765.
Created two issues for tracking the changes on Notary Project specifications:
/cc: @priteshbandi @Two-Hearts @gokarnm @shizhMSFT @toddysm @FeynmanZhou
Discussions are tracked in https://github.com/notaryproject/notation/discussions/767
Added more details for these two scenarios described in https://github.com/notaryproject/notation/issues/741#issuecomment-1644323623
@toddysm @gokarnm @iamsamirzon @priteshbandi @FeynmanZhou @Two-Hearts @shizhMSFT @sajayantony PTAL.
Use Notation CLI to sign the file on the file system. An example of command:
notatoin sign --blob <filepath> --media-type <type> --output-signature <signature_path>` --id <key_id> --plugin <plugin_name>
Upon successful signing, the siganture is stored on <signature_path>
mystore
, the type of trust store is ca
mypolicy.json
with the following content{
{
"name": "wabbit-networks-non-oci",
"signatureVerification": {
"level": "strict"
},
"trustStores": [ "ca:mystore" ],
"trustedIdentities": ["*"],
"scopes": [ "non-oci" ]
},
{
"name": "globalPolicy",
"signatureVerification": {
"level": "strict"
},
"trustStores": [ "ca:someGlobalCA" ],
"trustedIdentities": ["*"],
"scopes": [ "*" ]
}
}
notation policy
command to import trust policies from mypolicy.json
filenotation policy import mypolicy.json
notation verify
command to verify the file against the signaturenotation verify --blob <filepath> --media-type <media_type> --signature <signature_path> --policy-scope "non-oci"
<TODO: It is not to be solved by this feature, but it will be good to provide recommendation tooling for users to achieve this goal from e2e point of view.>
As discussed at the meeting on 1/2/2024, move this feature to next milestone 1.2.0
Move this to Future due to resource issue per community discussion.
Is your feature request related to a problem?
We would like to use Notation to sign and verify any arbitrary data and keep using the fine grain control it provides using trust policy and plugin model.
What solution do you propose?
Notation support sign and verify any arbitrary data
cc: @yizha1 @toddysm @shizhMSFT @patrickzheng200 @gokarnm @iamsamirzon