search

elastic / kibana

Your window into the Elastic Stack
https://www.elastic.co/products/kibana
Other
19.48k stars 8.04k forks source link

[ci/code-coverage] review coverage stats #65537

Open jbudz opened 4 years ago

jbudz commented 4 years ago

With https://github.com/elastic/kibana/pull/62336 we're going to have open visibility into code coverage. Before we go live I'd like to validate that stats going into the coverage report are an accurate representation. We may have some non standard reporting in our ftr mocha reports (e.g. we've sorta added a third pending state) that could skew.

I'd also like to take a close look at what the dashboard's representing. The magnitude here doesn't necessarily correlate with importance, e.g. coverage on security is a must and should bubble up. And probably an ascending sort on everything, framing on improvement areas but not as a cross team comparison.

elasticmachine commented 4 years ago

Pinging @elastic/kibana-qa (Team:QA)

elasticmachine commented 4 years ago

Pinging @elastic/kibana-operations (Team:Operations)

wayneseymour commented 4 years ago

@LeeDr hey boss, I reckon you'd like to be in the know on this as well

LeeDr commented 4 years ago

I created this draft PR https://github.com/elastic/kibana/pull/65582 with the ingest pipeline script I'm currently using to assign teams to the file paths in the coverage index. It's not currently used on the ingest of the data yet. So far I've just been running it to _update_by_query. But @wayneseymour will create another PR to add that to the ingest step after his main PR is merged.

I'm totally fine with changing visualizations any way the team deems appropriate. I just created a few and put them on a dashboard so I could see how we were coming along with getting the coverage job and ingest working. Maybe the best way to work on this is a zoom session?

I don't think I know how we'll validate the data (mocha coverage for example). We could add a checklist of team names in this issue for each team to "validate" the coverage data for their team?