tektoncd / results

Long term storage of execution results.
Apache License 2.0
77 stars 73 forks source link

Enable storing of Run Events as Record #748

Closed khrm closed 2 months ago

khrm commented 5 months ago

All Events related to taskrun or PipelineRun are stored when we are done with Runs and in a single List. This can be controlled by a flag passed to the watcher: "store-event". Setting it to false disables the storing of Events. The record Name of EventList is stored as results.tekton.dev/eventlist in TaskRun and PipelineRun.

Changes

Submitter Checklist

These are the criteria that every PR should meet, please check them off as you review them:

Release Notes

All Events related to taskrun or PipelineRun are stored when we are done with Runs and in a single List.
This can be controlled by a flag passed to the watcher: "store-event". Setting it to false 
disables the storing of Events. 
The record Name of EventList is stored as `results.tekton.dev/eventlist` in TaskRun and PipelineRun.
tekton-robot commented 5 months ago

The following is the coverage report on the affected files. Say /test pull-tekton-results-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/v1alpha2/types.go Do not exist 28.6%
pkg/apis/v1alpha2/types.go Do not exist 100.0%
pkg/apis/v1alpha2/types.go Do not exist 100.0%
pkg/apis/v1alpha2/types.go Do not exist 0.0%
pkg/apis/v1alpha2/types.go Do not exist 0.0%
pkg/watcher/reconciler/dynamic/dynamic.go 55.3% 53.6% -1.7
pkg/watcher/results/eventlist.go Do not exist 0.0%
khrm commented 5 months ago

/test pull-tekton-results-build-tests

tekton-robot commented 5 months ago

The following is the coverage report on the affected files. Say /test pull-tekton-results-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/v1alpha2/types.go Do not exist 28.6%
pkg/apis/v1alpha2/types.go Do not exist 100.0%
pkg/apis/v1alpha2/types.go Do not exist 100.0%
pkg/apis/v1alpha2/types.go Do not exist 0.0%
pkg/apis/v1alpha2/types.go Do not exist 0.0%
pkg/watcher/reconciler/dynamic/dynamic.go 55.3% 53.1% -2.2
pkg/watcher/results/eventlist.go Do not exist 0.0%
tekton-robot commented 5 months ago

The following is the coverage report on the affected files. Say /test pull-tekton-results-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/v1alpha2/types.go Do not exist 28.6%
pkg/apis/v1alpha2/types.go Do not exist 100.0%
pkg/apis/v1alpha2/types.go Do not exist 100.0%
pkg/apis/v1alpha2/types.go Do not exist 0.0%
pkg/apis/v1alpha2/types.go Do not exist 0.0%
pkg/watcher/reconciler/dynamic/dynamic.go 55.3% 56.5% 1.2
pkg/watcher/results/eventlist.go Do not exist 0.0%
khrm commented 5 months ago

/test pull-tekton-results-build-tests

tekton-robot commented 5 months ago

The following is the coverage report on the affected files. Say /test pull-tekton-results-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/v1alpha2/types.go Do not exist 28.6%
pkg/apis/v1alpha2/types.go Do not exist 100.0%
pkg/apis/v1alpha2/types.go Do not exist 100.0%
pkg/apis/v1alpha2/types.go Do not exist 0.0%
pkg/apis/v1alpha2/types.go Do not exist 0.0%
pkg/watcher/reconciler/dynamic/dynamic.go 55.3% 56.5% 1.2
pkg/watcher/results/eventlist.go Do not exist 0.0%
gabemontero commented 4 months ago

I'll take a look over the next day or two @khrm - thanks

khrm commented 2 months ago

/test pull-tekton-results-integration-tests

khrm commented 2 months ago

/test pull-tekton-results-integration-tests

khrm commented 2 months ago

/test pull-tekton-results-integration-tests

tekton-robot commented 2 months ago

The following is the coverage report on the affected files. Say /test pull-tekton-results-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/v1alpha3/types.go 100.0% 0.0% -100.0
pkg/apis/v1alpha3/types.go Do not exist 20.0%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 50.0%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/watcher/reconciler/dynamic/dynamic.go 54.5% 55.7% 1.3
pkg/watcher/results/eventlist.go Do not exist 0.0%
khrm commented 2 months ago

Apart from changing the default value of event_store to false. I have added a filter for values being stored. We don't store labels now. https://github.com/tektoncd/results/compare/ddf215448b8da116cff81ecb3f9a3d34239e1745..e1511ddb41d8a24f293d5d612b6f0575ffeb19fb

khrm commented 2 months ago

/test pull-tekton-results-integration-tests

tekton-robot commented 2 months ago

The following is the coverage report on the affected files. Say /test pull-tekton-results-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/v1alpha3/types.go Do not exist 20.0%
pkg/apis/v1alpha3/types.go Do not exist 66.7%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/watcher/reconciler/dynamic/dynamic.go 54.5% 55.8% 1.3
pkg/watcher/results/eventlist.go Do not exist 0.0%
tekton-robot commented 2 months ago

The following is the coverage report on the affected files. Say /test pull-tekton-results-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/v1alpha3/types.go Do not exist 28.6%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/watcher/reconciler/dynamic/dynamic.go 54.5% 55.6% 1.1
pkg/watcher/results/eventlist.go Do not exist 0.0%
khrm commented 2 months ago

/test pull-tekton-results-integration-tests

khrm commented 2 months ago

/test pull-tekton-results-integration-tests

tekton-robot commented 2 months ago

The following is the coverage report on the affected files. Say /test pull-tekton-results-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/v1alpha3/types.go Do not exist 16.7%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/watcher/reconciler/dynamic/dynamic.go 54.5% 55.6% 1.1
pkg/watcher/results/eventlist.go Do not exist 0.0%
khrm commented 2 months ago

/test pull-tekton-results-integration-tests

khrm commented 2 months ago

/test pull-tekton-results-integration-tests

khrm commented 2 months ago

@avinal We don't need a separate API. Events are stored as a record with a different type. If someone wants to query them, they can query based on Type.

tekton-robot commented 2 months ago

The following is the coverage report on the affected files. Say /test pull-tekton-results-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/v1alpha3/types.go Do not exist 16.7%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/watcher/reconciler/dynamic/dynamic.go 54.5% 56.0% 1.6
pkg/watcher/results/eventlist.go Do not exist 0.0%
khrm commented 2 months ago

/test pull-tekton-results-integration-tests

tekton-robot commented 2 months ago

The following is the coverage report on the affected files. Say /test pull-tekton-results-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/v1alpha3/types.go Do not exist 28.6%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/watcher/reconciler/dynamic/dynamic.go 54.5% 56.0% 1.6
pkg/watcher/results/eventlist.go Do not exist 0.0%
khrm commented 2 months ago

/test pull-tekton-results-build-tests

tekton-robot commented 2 months ago

The following is the coverage report on the affected files. Say /test pull-tekton-results-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/apis/v1alpha3/types.go Do not exist 16.7%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 100.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/apis/v1alpha3/types.go Do not exist 0.0%
pkg/watcher/reconciler/dynamic/dynamic.go 54.5% 56.2% 1.7
pkg/watcher/results/eventlist.go Do not exist 0.0%
enarha commented 2 months ago

The change is lgtm. One question, is it safe and have you tested switching that option back and forth, including reading the data? What I mean is start with the default (disabled), store some data, switch to enabled, store and read some data, switch to disabled, store and read some data (assuming keeping the storage and data).

tekton-robot commented 2 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: avinal, vdemeester

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/tektoncd/results/blob/main/OWNERS)~~ [avinal,vdemeester] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
khrm commented 2 months ago

@enarha I tested this by enabling and disabling it. It works in both cases. Older data works.

khrm commented 2 months ago

@avinal We need lgtm here. And then we can merge it.

enarha commented 2 months ago

/lgtm