Javascript data accurate?

madnight / githut

Github Language Statistics

https://madnight.github.io/githut

GNU Affero General Public License v3.0

967 stars 125 forks source link

Javascript data accurate? #89

Closed jamesmurdza closed 1 year ago

jamesmurdza commented 1 year ago

In the year 2021 Javascript pushes and PRs dropped dramatically in the year 2021. The scale of the change (40% down to 10% pushes) makes me think that this could be a data issue rather than a trend. Any explanation?

madnight commented 1 year ago

I think there are two reasons. One is the rise of typescript, which mostly gets it's user base from javascript. And the second thing is that I started filtering events like Pull Request from bot accounts like dependabot. There's a massive amounts of bot events in the JS world.

derwaldgeist commented 1 year ago

It would be nice if one could see the numbers of JavaScript and TypeScript combined in the graphics somehow, as it is basically the same ecosystem.

jamesmurdza commented 1 year ago

@madnight That makes sense. My hunch is that the change in counting method is the main difference. By other metrics, JavaScript activity is not declining that suddenly.

I guess there is no way to retroactively get trend data?

madnight commented 1 year ago

The Problem is that I haven't yet removed all bot accounts for the whole history, because that requires quite a bit of BigQuery query volume. I think as soon as I apply it to the whole history the trend data should be correct again.

madnight commented 1 year ago

We also have a problem with pushes regarding Python.

The following filter:

WHERE NOT LOWER(actor.login) LIKE "%bot%")

is not good enough, because there are scripted actors that generate huge amount of pushes without "bot" in their name. I updated the query again and try a new approach. Any individual actor.login (github user) can only generate 1000 pushes per quarter per language, otherwise it will not be counted for the this statistic, since we are interested in language popularity among the 99.9% user base and not in the popularity of pushes of bots and scripts.

But for this month I've used all my quota:

Error while querying the BigQuery Google API Error: Quota exceeded: Your project exceeded quota for free query bytes scanned. For more information, see https://cloud.google.com/bigquery/docs/troubleshoot-quotas

madnight commented 1 year ago

Okay, I hade enough quota this time and was able to fix the issue.

Before:

After:

The data is now much more accurate in terms of reflecting popularity, since it does not take bots into account. As we can see JS Pull Requests have been inflated for quite some times, due to bots like dependabot.