search

kubernetes / test-infra

Test infrastructure for the Kubernetes project.
Apache License 2.0
3.82k stars 2.63k forks source link

Add presubmits in testgrid for repos/modules #18159

Closed helenfeng737 closed 2 years ago

helenfeng737 commented 4 years ago

Below are all repos that don't have their presubmits in testgrid. This issue is to track moving this down to just kubernetes-security/

var noPresubmitsInTestgridPrefixes = []string{
    "containerd/cri",
    "GoogleCloudPlatform/k8s-multicluster-ingress",
    "kubeflow/pipelines",
    "kubernetes-sigs/cluster-capacity",
    "kubernetes-sigs/gcp-compute-persistent-disk-csi-driver",
    "kubernetes-sigs/gcp-filestore-csi-driver",
    "kubernetes-sigs/kind",
    "kubernetes-sigs/kubebuilder-declarative-pattern",
    "kubernetes-sigs/scheduler-plugins",
    "kubernetes-sigs/service-catalog",
    "kubernetes-sigs/sig-storage-local-static-provisioner",
    "kubernetes-sigs/slack-infra",
    "kubernetes-sigs/testing_frameworks",
    "kubernetes/client-go",
    "kubernetes/cloud-provider-openstack",
    "kubernetes/dns",
    "kubernetes/enhancements",
    "kubernetes/ingress-gce",
    "kubernetes/kubeadm",
    "kubernetes/minikube",
    // This is the one entry that should be here
    "kubernetes-security/",
}

This used to be tracked in the code. Make it an independent issue for easier tracking.
https://github.com/kubernetes/test-infra/blob/master/config/tests/testgrids/config_test.go

helenfeng737 commented 4 years ago

/cc @spiffxp

BenTheElder commented 4 years ago

I'm not sure why we'd want presubmits in testgrid. Presubmits are very noisy and the heatmap in prow gives better insight (e.g. PRs that don't build are clearly distinct by runtime)

BenTheElder commented 4 years ago

cc @spiffxp

chases2 commented 3 years ago

I got tripped up on this as well. Why do we want to display these tests on TestGrid? They run at variable frequencies against code that may not actually merge or work. Is that grid useful?

spiffxp commented 3 years ago

It's about ensuring that testgrid can be a consistent gateway to all jobs executed by prow.k8s.io, instead of sometimes having to go view prow's deck, or spyglass' job-history page.

Presubmits aren't guaranteed to pass reliably, but testgrid can still provide a useful visual signal if one set of tests starts failing consistently across presubmits

chases2 commented 3 years ago

We discussed this during today's sig-testing meeting. We didn't come to a consensus, but I do want to note what I heard:

For adding all presubmits to TestGrid:

Against requiring presubmits in TestGrid:

fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

spiffxp commented 3 years ago

/remove-lifecycle stale

Against requiring presubmits in TestGrid:

  • Tests may run very frequently, putting strain on TG's frontend

Presubmits should default to shorter retention to avoid this strain

-Tuning staleness and pass/fail thresholds for alerting is too difficult to get good signal out of

Maybe true, but I still think testgrid's summary view can provide me a better glance of "how are merge-blocking presubmits doing" vs looking at https://prow.k8s.io/?repo=kubernetes%2Fkubernetes&type=presubmit at the moment:

I think between configurator getting a little smarter about what kind of information is available from prow config and prow jobs, and possibly supporting multiple defaults, there's room for improvement.

fejta-bot commented 3 years ago

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle stale

fejta-bot commented 3 years ago

Stale issues rot after 30d of inactivity. Mark the issue as fresh with /remove-lifecycle rotten. Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community. /lifecycle rotten

BenTheElder commented 3 years ago

I'm not sure we should enforce presubmits in testgrid, I think it would be reasonable to just exclude presubmits from this check and require it for periodic / postsubmit

Periodic / postsubmit we don't want to run if nobody is watching. But presubmits report to PRs.

k8s-triage-robot commented 3 years ago

Rotten issues close after 30d of inactivity. Reopen the issue with /reopen. Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community. /close

k8s-ci-robot commented 3 years ago

@k8s-triage-robot: Closing this issue.

In response to [this](https://github.com/kubernetes/test-infra/issues/18159#issuecomment-889488451): >Rotten issues close after 30d of inactivity. >Reopen the issue with `/reopen`. >Mark the issue as fresh with `/remove-lifecycle rotten`. > >Send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). >/close Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
spiffxp commented 3 years ago

/reopen /remove-lifecycle rotten If we shouldn't enforce this, we should drop the test. Or, finish implementing so there are no more special cases. This middle ground is complexity that doesn't need to be present.

spiffxp commented 3 years ago

/sig testing

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

k8s-triage-robot commented 2 years ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close

k8s-ci-robot commented 2 years ago

@k8s-triage-robot: Closing this issue.

In response to [this](https://github.com/kubernetes/test-infra/issues/18159#issuecomment-1007626709): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues and PRs according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue or PR with `/reopen` >- Mark this issue or PR as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.