[bug] The kubernetes sig activities seems not right

kerthcet commented 2 months ago

See https://k8s.devstats.cncf.io/d/13/developer-activity-counts-by-repository-group?orgId=1&var-period_name=Last%20year&var-metric=contributions&var-repogroup_name=SIG%20Scheduling&var-repo_name=kubernetes%2Fkubernetes&var-country_name=All for kubernetes sig-scheduling

However, as sig-scheduling maintainer, I think the graph not accurate. Can this be fixed which will help a lot to observe the sig health state.

lukaszgryglicki commented 2 months ago

If you think this is wrong, you need to tell me what is wrong there, so I can fix...

lukaszgryglicki commented 2 months ago

I may check on Friday/Monday, but due to a lot of work accumulation during my PTO this can be postponed a bit - but this is on my dashboard FYI.

kerthcet commented 2 months ago

Well, since this dashboard is the only way to see the metrics, I can't say in an official way, but what I can say is some people like alculquicondor or sanposhiho should definitely be in the list, they're active contributors.

kerthcet commented 2 months ago

IIUC, this is the dashboard for kubernetes projects only. Not including subprojects.

lukaszgryglicki commented 2 months ago

Will check this next week - the two uses mentioned (I'll check what I have in the database vs. dashboards for specific SIG repo group).

kerthcet commented 2 months ago

Thanks!

lukaszgryglicki commented 2 months ago

It is for Kubernetes project (which is tracking the following GitHub orgs/repos) (link):

kubernetes,kubernetes-client,kubernetes-incubator,kubernetes-csi,
kubernetes-graveyard,kubernetes-incubator-retired,kubernetes-sig-testing,
kubernetes-providers,kubernetes-addons,kubernetes-retired,
kubernetes-extensions,kubernetes-federation,kubernetes-security,
kubernetes-sidecars,kubernetes-tools,kubernetes-test,kubernetes-sigs

Excluding repos (link):

kubernetes/api,kubernetes/apiextensions-apiserver,kubernetes/apimachinery,\
kubernetes/apiserver,kubernetes/client-go,kubernetes/code-generator,\
kubernetes/kube-aggregator,kubernetes/metrics,kubernetes/sample-apiserver,\
kubernetes/sample-controller,kubernetes/helm,kubernetes/deployment-manager,\
kubernetes/charts,kubernetes/cli-runtime,kubernetes/csi-api,\
kubernetes/kube-proxy,kubernetes/kube-controller-manager,\
kubernetes/kube-scheduler,kubernetes/kubelet,kubernetes/sample-cli-plugin,\
kubernetes/application-dm-templates,kubernetes/cluster-bootstrap,\
kubernetes/cloud-provider,kubernetes-sigs/cri-o,kubernetes-incubator/ocid,\
kubernetes-incubator/cri-o

And you specifically requested data for SIG-Scheduling which means those repos/orgs (link):

kubernetes-incubator/cluster-capacity
kubernetes-incubator/descheduler
kubernetes-incubator/kube-arbitrator
kubernetes-incubator/rescheduler
kubernetes-sigs/cluster-capacity
kubernetes-sigs/descheduler
kubernetes-sigs/kube-batch
kubernetes-sigs/poseidon
kubernetes-sigs/scheduler-plugins
kubernetes-sigs/scheduling_poseidon
kubernetes-sigs/volcano

Checking this...

lukaszgryglicki commented 2 months ago

So I did some investigation and those are contributions for those two users in SIG Scheduling repo group only:

gha=# select r.name, e.dup_actor_login as login, min(e.created_at) as first, max(e.created_at) as last, count(*) as events from gha_events e, gha_repos r where e.repo_id = r.id and e.dup_repo_name = r.name and r.repo_group = 'SIG Scheduling' and e.dup_actor_login in ('alculquicondor', 'sanposhiho') and e.type in ('PushEvent', 'PullRequestEvent', 'IssuesEvent', 'PullRequestReviewEvent', 'CommitCommentEvent', 'IssueCommentEvent', 'PullRequestReviewCommentEvent') group by 1, 2 order by 1, 2;
               name                |     login      |        first        |        last         | events 
-----------------------------------+----------------+---------------------+---------------------+--------
 kubernetes-sigs/descheduler       | alculquicondor | 2020-02-28 15:18:00 | 2022-09-07 15:43:05 |      3
 kubernetes-sigs/descheduler       | sanposhiho     | 2022-10-06 05:49:09 | 2022-10-06 05:49:41 |      2
 kubernetes-sigs/kube-batch        | alculquicondor | 2021-02-12 14:00:27 | 2021-02-12 14:00:27 |      1
 kubernetes-sigs/scheduler-plugins | alculquicondor | 2020-04-03 13:25:54 | 2021-08-17 19:19:59 |     94
 kubernetes-sigs/scheduler-plugins | sanposhiho     | 2021-11-25 23:18:57 | 2022-02-15 15:20:41 |     12
(5 rows)

This repo group only contains data from those repos:

gha=# select distinct name from gha_repos where repo_group = 'SIG Scheduling';
                 name                  
---------------------------------------
 kubernetes-incubator/kube-arbitrator
 kubernetes-sigs/cluster-capacity
 kubernetes-incubator/cluster-capacity
 kubernetes-sigs/descheduler
 kubernetes-sigs/scheduling_poseidon
 kubernetes-sigs/poseidon
 kubernetes-sigs/volcano
 kubernetes-sigs/scheduler-plugins
 kubernetes-sigs/kube-batch
 kubernetes-incubator/rescheduler
 kubernetes-incubator/descheduler
(11 rows)

If I aggregate for all repos I will get:

gha=# select e.dup_actor_login as login, date(min(e.created_at)) as first, date(max(e.created_at)) as last, count(*) as events from gha_events e, gha_repos r where e.repo_id = r.id and e.dup_repo_name = r.name and r.repo_group = 'SIG Scheduling' and e.dup_actor_login in ('alculquicondor', 'sanposhiho') and e.type in ('PushEvent', 'PullRequestEvent', 'IssuesEvent', 'PullRequestReviewEvent', 'CommitCommentEvent', 'IssueCommentEvent', 'PullRequestReviewCommentEvent') group by 1 order by 1;
     login      |   first    |    last    | events 
----------------+------------+------------+--------
 alculquicondor | 2020-02-28 | 2022-09-07 |     98
 sanposhiho     | 2021-11-25 | 2022-10-06 |     14
(2 rows)

So alculquicondor has 98 contributions between 2020-02-28 and 2022-09-07 while sanposhiho has 14 contributions between 2021-11-25 and 2022-10-06.

When I go here I can see alculquicondor has 98 contributions:

And sanposhiho 14 - exactly as I see on the database:

SO mentioned users are listed, but you need to select the correct time range - they stopped contributing around 9/22 and 10/22 respectively.

If there is still something wrong, let me know what - I will investigate.

And those are exact events that I have in the database:

gha=# select e.dup_actor_login as login, e.created_at, r.name, e.type from gha_events e, gha_repos r where e.repo_id = r.id and e.dup_repo_name = r.name and r.repo_group = 'SIG Scheduling' and e.dup_actor_login in ('alculquicondor', 'sanposhiho') and e.type in ('PushEvent', 'PullRequestEvent', 'IssuesEvent', 'PullRequestReviewEvent', 'CommitCommentEvent', 'IssueCommentEvent', 'PullRequestReviewCommentEvent') order by 1, 2;
     login      |     created_at      |               name                |             type              
----------------+---------------------+-----------------------------------+-------------------------------
 alculquicondor | 2020-02-28 15:18:00 | kubernetes-sigs/descheduler       | IssueCommentEvent
 alculquicondor | 2020-04-03 13:25:54 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-04-03 13:28:08 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-04-03 13:29:14 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-04-03 13:34:11 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-04-03 13:41:01 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-04-03 13:41:50 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-04-03 13:45:25 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-04-03 13:46:05 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-04-03 19:29:16 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-04-03 19:32:43 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-04-03 19:45:56 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-04-03 19:55:40 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-04-06 23:03:44 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-04-30 20:02:47 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-05-04 14:00:05 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-04 14:05:53 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-04 14:07:10 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-04 14:08:50 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-04 14:12:17 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-04 14:14:47 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-04 14:17:10 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-04 14:19:54 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-04 14:47:06 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-04 17:09:13 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-04 19:40:55 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-05 17:05:16 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-06 14:17:14 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-06 14:19:06 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-06 14:25:59 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-06 15:44:38 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-06 15:45:23 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-06 16:59:08 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-05-06 17:00:21 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-05-07 15:15:04 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-05-12 19:52:55 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-06-26 18:24:49 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-06-26 20:21:19 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-06-26 20:52:23 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-10-19 13:25:55 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-10-20 15:05:22 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 15:07:45 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 15:15:36 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 15:16:43 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 15:19:46 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 15:21:57 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 15:22:46 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 15:24:09 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 15:33:41 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 15:33:46 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-20 15:33:46 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-20 16:41:00 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 16:41:01 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-20 16:41:50 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 16:41:51 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-20 16:49:12 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 16:49:13 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-20 17:14:44 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 17:14:45 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-20 18:08:05 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 18:08:05 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-20 18:37:07 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 18:37:07 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-20 18:39:49 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-20 18:39:49 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 19:54:18 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 19:54:19 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-20 22:36:07 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-20 22:36:08 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-21 16:18:00 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-10-23 13:37:22 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-23 13:39:00 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-23 13:40:47 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-23 13:50:47 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-23 13:52:26 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-23 13:58:28 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-23 13:58:28 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-23 21:02:44 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-23 21:02:45 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-23 21:04:33 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-23 21:04:33 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-26 15:09:35 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-26 15:09:44 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-26 15:09:44 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-26 17:05:53 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-26 17:05:53 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-26 17:43:38 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2020-10-26 17:43:38 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2020-10-26 23:09:10 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2020-10-26 23:37:11 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2021-02-12 14:00:27 | kubernetes-sigs/kube-batch        | IssueCommentEvent
 alculquicondor | 2021-04-16 17:29:07 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2021-04-19 13:41:06 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2021-08-12 20:24:07 | kubernetes-sigs/scheduler-plugins | PullRequestReviewEvent
 alculquicondor | 2021-08-12 20:24:07 | kubernetes-sigs/scheduler-plugins | PullRequestReviewCommentEvent
 alculquicondor | 2021-08-17 19:19:59 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 alculquicondor | 2022-09-07 15:18:55 | kubernetes-sigs/descheduler       | IssueCommentEvent
 alculquicondor | 2022-09-07 15:43:05 | kubernetes-sigs/descheduler       | IssueCommentEvent
 sanposhiho     | 2021-11-25 23:18:57 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 sanposhiho     | 2022-02-10 10:50:04 | kubernetes-sigs/scheduler-plugins | PullRequestEvent
 sanposhiho     | 2022-02-10 10:54:48 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 sanposhiho     | 2022-02-10 10:54:58 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 sanposhiho     | 2022-02-10 13:18:35 | kubernetes-sigs/scheduler-plugins | PullRequestEvent
 sanposhiho     | 2022-02-10 13:19:17 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 sanposhiho     | 2022-02-10 13:22:05 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 sanposhiho     | 2022-02-10 14:08:17 | kubernetes-sigs/scheduler-plugins | PullRequestEvent
 sanposhiho     | 2022-02-10 14:12:07 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 sanposhiho     | 2022-02-14 09:42:39 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 sanposhiho     | 2022-02-14 09:46:40 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 sanposhiho     | 2022-02-15 15:20:41 | kubernetes-sigs/scheduler-plugins | IssueCommentEvent
 sanposhiho     | 2022-10-06 05:49:09 | kubernetes-sigs/descheduler       | PullRequestEvent
 sanposhiho     | 2022-10-06 05:49:41 | kubernetes-sigs/descheduler       | IssueCommentEvent
(112 rows)

lukaszgryglicki commented 2 months ago

Closing, please reopen if needed.

kerthcet commented 2 months ago

Thanks for the digging, really appreciate! So right now, I have two questions:

The first one is about the: And you specifically requested data for SIG-Scheduling which means those repos/orgs: . These repos changed a lot as we archived several ones and also created new repos, so how these projects synced with the kubernetes community? Manual?
The second one is we quite care about the contributors to the kubernetes project, which means the kubernetes/kubernetes repo only, not churning with other projects, and as you know, each PR/issue is classified by different SIGs, so maybe that's possible?

lukaszgryglicki commented 2 months ago

To update SIG definition just create a PR updating list of repos for file that I've mentioned in my link.
Not quite get what you need exactly, in my link you have a list of repos which go to SIG Scheduling - there are more repo groups for other SIGs as well as k/k repo which goes to Kubernetes repository group.

Which repo goes where is a community decisions, there already been several PRs add/removing/modifying SIGs and they are reviewed, eventually approved and once that happens I'm regenerating all stats as this also changes past-data assignment between different repo groups.

cncf / devstats

[bug] The kubernetes sig activities seems not right #63