Closed leonardpahlke closed 1 year ago
cc k/sig-contribex chairs @alisondy @mrbobbytables cc k/sig-release chairs @justaugustus @saschagrunert
I don't see anything too bad in there if those are the only types of reports being generated, but to be sure I'd ask the applicants to opt-in to sharing their aggregated data publicly and list which fields would be included in that.
/assign @jberkus
@jberkus do you have any thoughts or see anything that might be an issue?
but to be sure I'd ask the applicants to opt-in to sharing their aggregated data publicly and list which fields would be included in that.
yes! see: https://github.com/kubernetes-sigs/release-team-shadow-stats/issues/10
Are we asking about past shadows, or future applicants? Because those are two different situations.
For future applicants
OK. My comments below. We'll also want to have CoCC take a look to see if there are potential diversity issues.
Also, we're going to have some new questions in the new shadow application, so we'll want charts for those.
Also, we're going to have some new questions in the new shadow application, so we'll want charts for those.
+1
We'll also want to have CoCC take a look to see if there are potential diversity issues.
cc CoCC Members: @celestehorgan @cpanato @karenhchu @palnabarun @vllry
tag @reylejano to follow
In my opinion, we should only store/publish the anonymized reports on GitHub e.g. https://github.com/kubernetes-sigs/release-team-shadow-stats/tree/main/examples/plots
My concern is that we currently have non-anonymized data in https://github.com/kubernetes-sigs/release-team-shadow-stats/tree/main/examples/applicants
cc: @jeremyrickard @cici37 @JamesLaverack @jrsapi
Yeah, so we couldn't put that data into GitHub.
The data in https://github.com/kubernetes-sigs/release-team-shadow-stats/tree/main/examples/applicants is just fake data isn't it?
100% agree what we should only store/publish anonymized reports and should not store things like the examples data (which i am still assuming is just example fake data)
My concern is that we currently have non-anonymized data in examples/applicants
There are no plans to make these md files publicly available. These files are generated for a different usecase. They should only be made available to the release team role leads to help reading through the shadow application files - they are not part of the report. Only the diagrams/plots should be released.
@jeremyrickard yes, it's dummy / fake data :)
@leonardpahlke Thanks for clearing up about the data in https://github.com/kubernetes-sigs/release-team-shadow-stats/tree/main/examples/applicants
Perhaps we should add a small README.md in there that states the data is fake
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle stale
/lifecycle rotten
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
@leonardpahlke what are we currently doing on this report?
@jberkus Currently, most efforts are on hold. I have been doing some work behind the scenes to test out some libraries, but that is not yet ready to get pushed. I will have time in a bit over a week to continue with this and close some more issues. (still planning to wrap it all up in 1.25)
/remove-lifecycle stale
@jberkus The charts for 1.26 are ready (see examples with dummy data https://github.com/kubernetes-sigs/release-team-shadow-stats/tree/main/examples/plots). If an applicant was not opted in, the data in the charts will be ignored. If there are two or fewer applicants per category (e.g., 2 applicants who chose the US East time zone), the data in the charts will be ignored - US East will not appear in this case (see the concerns mentioned earlier).
Is there any concern about sharing this information (the charts) in a report with the community?
No, that was the purpose of the charts and why we got a release. The only data that was concerning (pronouns) has been removed.
Some specific critique on the current versions of the charts:
Two more general comments:
Finally: I'd like a data dump to be available on request with PII removed. Because there's other ad-hoc manipulations that folks might want to do.
Thanks, @jberkus, for your feedback & input :)
- able to attend burndown meetings: the labels are getting cut off, and aren't understandible
- able to attend release team meetings: same
- previous roles: what even is this question? You'll need to expand the title so it's clear what's being measured.
Agree, formatting & rephrasing is needed here 👍
timezone: this pie chart is still really hard to make any sense of, particularly since applicants were able to fill in ad-hoc entries. We'd need to normalize the data, ...
Right, normalizing data is something we should improve in future shadow application forms, this would make many things a lot easier :D (added to the 1.26 retro Agenda).
... and then probably render it in a bar chart in time zone order, for it to be useful
Yes, I also discussed this with @palnabarun (selecting the right chart to transport the message). I was thinking about a diagram in map / earth format for the timezone, the option with bar charts is also good (maybe a bit easier to implement, too).
It would be useful if each chart also had a table of numerical data
Interesting idea, I can see that this view can be useful. Perhaps this can be placed in the report next to the chart.
One of the goals here is to compare who applied with who was accepted. So all of these charts need to filter applied/accepted
We don't have this information yet, because we "only" process the application form, the shadow selection comes afterwards (so we need manual input to display this information in charts).
Question/concern: do we really want to display this? How might this look in a chart?
IMHO, processing of accepted applicants could be something for the future. We can already see the "status of the shadow application program" from the above charts (and maybe some similar charts), which is the goal of the report (at least so far).
I'd like a data dump to be available on request with PII removed. Because there's other ad-hoc manipulations that folks might want to do.
Are you thinking of the raw data which is used to generate the diagrams? (JSON format or smth similar). We could publish this alongside the report. +1
Right, normalizing data is something we should improve in future shadow application forms, this would make many things a lot easier :D (added to the 1.26 retro Agenda).
You should modify the form also. But in the meantime we should clean up the data by hand.
IMHO, processing of accepted applicants could be something for the future. We can already see the "status of the shadow application program" from the above charts (and maybe some similar charts), which is the goal of the report (at least so far).
Sure, this will have to be hand-entered. Shouldn't take that long. But the primary reason to analyze this data in the first place was to check for selection bias, no?
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.
This bot triages issues and PRs according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle stale
/lifecycle rotten
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale
fix labels are getting cut off: https://github.com/kubernetes-sigs/release-team-shadow-stats/pull/19
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle stale
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/remove-lifecycle rotten
/close
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.
This bot triages issues according to the following rules:
lifecycle/stale
is appliedlifecycle/stale
was applied, lifecycle/rotten
is appliedlifecycle/rotten
was applied, the issue is closedYou can:
/reopen
/remove-lifecycle rotten
Please send feedback to sig-contributor-experience at kubernetes/community.
/close not-planned
@k8s-triage-robot: Closing this issue, marking it as "Not Planned".
To increase transparency about and improve the release-team's shadow program, we would like to make some information publicly available (in a report format at the start of each release cycle). See the examples/plots folder for examples of the information that should be included in a report. Because we have a responsibility to ensure that applicants entrust us with their information, we must ensure that we do not share personal information. This needs to be coordinated with contribex.
What we need to clarify with contribex:
see tracking issue: https://github.com/kubernetes-sigs/release-team-shadow-stats/issues/4