search

kubernetes-sigs / krew

📦 Find and install kubectl plugins
https://krew.sigs.k8s.io
Apache License 2.0
6.37k stars 369 forks source link

Proposal: implement a manifest style validation tool for krew-index specifically #586

Closed ahmetb closed 3 years ago

ahmetb commented 4 years ago

Currently, our plugin manifest validation functionality which is in the YAML parsing path has some style/cosmetic checks that have no effect on plugin installation. This is a result of not separating "cosmetic validation" and "functional validation".

If we continue on this trajectory, we will be adding more "cosmetic validation" in future versions, which will break older plugin manifests. So we can't do that. Furthermore, with custom plugin indexes (#566), we cannot control the plugin manifests out in the wild –but we have to be able to continue working with them.

I propose that we strip off these "cosmetic validations" from the logic in krew machinery, and create a krew_index_manifest_validator tool, specifically for krew-index that runs as part of its CI for each PR.

This would help reduce code review toil we have been manually enforcing in krew-index, as well as separate structural validation vs cosmetic validation, such as:

The problem is: We already have some validation in cmd/validate-krew-manifest tool that is run only for krew-index. This validation:

  1. runs all structural validation through Validate()
  2. ensures: all spec.platforms[] items are used
  3. ensures: no overlapping spec.platform[].selectors
  4. ensures: all spec.platforms[] install fine
  • point 1: Already validated by krew, during parsing
  • point 2: Not a functional validation, if there's unused platforms[], krew continues to work fine.
  • point 3: I believe Krew continues to work fine when there are overlapping selectors –though, the behavior is undefined (not sure which one would be picked up, as we don't guarantee it). We could move this to "functional validation", though I'm not sure we totally should.
  • point 4: Not "structural validation", just ensures plugins merged to krew-index are installable, we can't enforce this in custom indexes.

So maybe we just make cmd/validate-krew-manifest more about "styling" and make it clear it's meant for krew-index.

/kind proposal /cc @corneliusweig

corneliusweig commented 4 years ago

This is a sound plan. The only thing I'd add is that we strive to build our krew-index manifest validation tool in a way that it can be re-used by other projects. After all, everybody should be able to easily check what we think are good practices for plugin manifests.

ahmetb commented 4 years ago

yeah. I think it would be reusable by public indexes out of the box. private indexes have different requirements (no LICENSE etc).

ahmetb commented 4 years ago

tl;dr I'm thinking we should rename the existing tool to krew_index_manifest_validator and extend its capabilities to more cosmetics validation.

fejta-bot commented 4 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

fejta-bot commented 4 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

corneliusweig commented 4 years ago

/remove-lifecycle rotten

fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

corneliusweig commented 3 years ago

/remove-lifycycle stale /freeze I think this is important long-term and we should work on this eventually.

fejta-bot commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle rotten

fejta-bot commented 3 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /close

k8s-ci-robot commented 3 years ago

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/krew/issues/586#issuecomment-760305783): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-testing, kubernetes/test-infra and/or [fejta](https://github.com/fejta). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
corneliusweig commented 3 years ago

/reopen /remove-lifecycle rotten /good-first-issue

k8s-ci-robot commented 3 years ago

@corneliusweig: Reopened this issue.

In response to [this](https://github.com/kubernetes-sigs/krew/issues/586#issuecomment-766455969): >/reopen >/remove-lifecycle rotten >/good-first-issue Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

fejta-bot commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

fejta-bot commented 3 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community. /close

k8s-ci-robot commented 3 years ago

@fejta-bot: Closing this issue.

In response to [this](https://github.com/kubernetes-sigs/krew/issues/586#issuecomment-867242445): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.