search

slsa-framework / slsa-verifier

Verify provenance from SLSA compliant builders
Apache License 2.0
231 stars 50 forks source link

[doc] Add README section on semver #431

Open ianlewis opened 1 year ago

ianlewis commented 1 year ago

Let's add a section in the README to call out which parts of the code follow semver: the CLI arguments and behavior. Currently no guarantees on CLI output and API.

asraa commented 1 year ago

YES! If we are OK with this I'd love to enhance the package API: https://github.com/slsa-framework/slsa-verifier/issues/422

@behnazh-w curious if you have feedback here. I've been holding off on library updates because of that, but if we only want to guarantee CLI for now, would be very happy to do more cleanup.

ianlewis commented 1 year ago

I don't think there are many users of the verifier API. The only public one seems to be https://github.com/ddworken/hishtory by @ddworken. @suzuki-shunsuke also created #373 to fix go install so maybe they are using it?

@ddworken @suzuki-shunsuke Do you have any thoughts?

To be fair, it's also not a huge burden to just bump the semver but we'd want to minimize the number of times we have to bump so we'd want to do all the changes to the API in one go. Probably as part of the API updates for the "slsa-verifier as a service".

suzuki-shunsuke commented 1 year ago

aqua, which is a declarative CLI Version Manager I maintain, uses slsa-verifer as Go library. https://github.com/aquaproj/aqua/blob/853bccc0d0b8f4a76e1007807a75c8db630d6565/go.mod#L22 https://github.com/aquaproj/aqua/blob/853bccc0d0b8f4a76e1007807a75c8db630d6565/pkg/slsa/verifier.go#L69-L79 Currently slsa-vefier doesn't release pre built binaries except for linux/amd64, but if slsa-verifier will release pre built binaries for major platforms such as Windows, macOS, Linux / amd64, arm64, I would like to replace Go library to pre built binaries. So I created a pull request to release pre built binaries for multiple platforms #421 .

ref. https://aquaproj.github.io/docs/reference/cosign-slsa https://twitter.com/aquaclivm/status/1607714636039876610

ianlewis commented 1 year ago

@suzuki-shunsuke Got it thanks!

Hmm, Strange. If you have it listed in your go.mod, I wonder why aqua doesn't show up in the dependents for slsa-verifier... https://github.com/slsa-framework/slsa-verifier/network/dependents?dependent_type=REPOSITORY

suzuki-shunsuke commented 1 year ago

Repositories that depend on github.com/slsa-framework/slsa-verifier/v2

aqua depends on not github.com/slsa-framework/slsa-verifier but github.com/slsa-framework/slsa-verifier/v2.

https://github.com/slsa-framework/slsa-verifier/network/dependents?package_id=UGFja2FnZS0zNDgxMTc0MDc3

image
ianlewis commented 1 year ago

Repositories that depend on github.com/slsa-framework/slsa-verifier/v2

Aha, I didn't realize they split it up by versioned package name.

behnazh-w commented 1 year ago

@asraa I think the main question would be whether you would like the slsa-verifier to be used with its CLI interface or as a library. I would prefer slsa-verifier to be mainly available as a library because it gives a lot of flexibility to end users. In this case the library would be versioned and the CLI would be just a wrapper that is always compatible with the library APIs.

ianlewis commented 1 year ago

@asraa I think the main question would be whether you would like the slsa-verifier to be used with its CLI interface or as a library. I would prefer slsa-verifier to be mainly available as a library because it gives a lot of flexibility to end users. In this case the library would be versioned and the CLI would be just a wrapper that is always compatible with the library APIs.

I think that we would like to go in the direction of supporting it as an API. Whether we should support it as an API now though is perhaps the question.

I'm not sure we ever intended to provide the current API as stable API and I think we may like to do some more iteration before it becomes more stable but I think we can stick to semver without it being too burdensome.

laurentsimon commented 1 year ago

+1 to what @ianlewis said. We want to revamp the API and have the CLI just "wrap" it https://github.com/slsa-framework/slsa-verifier/issues/46

There are only a handful of users who use the API, so it should be OK to make the API experimental and subject to changes until we have the issue above resolved. As @ianlewis said, we did not intend for the API to be stable atm. Our mistake for not making this more explicitly in the README.

@suzuki-shunsuke , we can work with you when we update it to help you migrate. Would that be OK? Let's create a new milestone for it as well.

Would that work for everyone?

re: semver Section in README. Let's also call out that CLI arguments marked as [experimental] don't follow semver, and that builders with no documentation, marked as WIP or experimental don't follow semver either. We should probably turn the list of builders / generators into a table to make it easier for users to read.

suzuki-shunsuke commented 1 year ago

Let's also call out that CLI arguments marked as [experimental] don't follow semver

Hmm. CLI users wants CLI to follow semver. Go library is useful, but CLI is also useful. SLSA Provenance doesn't depend on Go, so CLI is better for non Go users. How about maitaining CLI in the other repository like slsa-verifer-cli and following semver?

laurentsimon commented 1 year ago

We do follow semver for CLI. There are sometimes some new arguments we label as [experimental] before officially releasing the new feature.

suzuki-shunsuke commented 1 year ago

Oh, I see. I misunderstood. Thank you for your explanation.

ddworken commented 1 year ago

Sorry for the late reply, but I'll chime in to say that I am using slsa-verifier as an API for hiSHtory. It is used to check that updates are valid. From my POV, having breaking changes to the API isn't a huge deal--I'm happy to tweak my code to conform to the newest API as needed. I just ask that the number of breaking changes to attestations are kept to a minimum, since those can break updates for my tool.