SAP-samples / artifact-of-the-month

A SAPUI5 application that ranks the popularity of SAP packages and repositories from multiple platforms. The usage data is pulled periodically from these platforms by GitHub Actions. The web app itself is hosted via GitHub Pages.
https://sap-samples.github.io/artifact-of-the-month/
Apache License 2.0
9 stars 9 forks source link

new Github Action: Update only modifiable data from GitHub #14

Closed marianfoo closed 2 years ago

marianfoo commented 2 years ago

As discussed in #9 I tried to create a solution. I created an action and a new file 'update-gh.ts'.

I have run it once in my fork: https://github.com/marianfoo/artifact-of-the-month/commit/49f8a2f6fb285f47b38c22a4982dcd04ae0148b0

IObert commented 2 years ago

Thanks a lot for this PR 🚀 I added one comment, the rest looks good to me.

But I do have one more ask: Can you change the branch of your Github Page (https://marianfoo.github.io/artifact-of-the-month/#/new) to "update-only-github" so that I can see this in action?

marianfoo commented 2 years ago

Hi Marius, Thanks for the feedback. I'll try to set up an example again. The action always included commits in the PR branch. I then tested too much and hit the github rate limit.

marianfoo commented 2 years ago

Hi @IObert , here is the current example https://marianfoo.github.io/artifact-of-the-month/?sap-language=en#/updated Unfortunately with the new sorting, the ranks are bit confusing, right? Is it may be better here to not show the ranks at all? image

IObert commented 2 years ago

Yes, I think this would be too confusing. I would rather prefer to update only the UI5 tags but not the rank.

WDYT?

marianfoo commented 2 years ago

Right. The purpose of this PR is specifically not to update the ranks. And the page "updated" should simply show the top 99 in descending order after the last update.

marianfoo commented 2 years ago

Hi @IObert still have a problem. I can´t make it the Github Request work again. Everytime i try to run it, there is a error that i´ve exceeded the rate limit. I´ve waited more than a day but i still throws this error. https://github.com/marianfoo/artifact-of-the-month/runs/4288494034?check_suite_focus=true (node:1608) UnhandledPromiseRejectionWarning: HttpError: You have exceeded a secondary rate limit. Please wait a few minutes before you try again. at /home/runner/work/artifact-of-the-month/artifact-of-the-month/node_modules/@octokit/request/dist-src/fetch-wrapper.js:68:27 at processTicksAndRejections (internal/process/task_queues.js:95:5)

Any ideas?

IObert commented 2 years ago

Strange, sounds to me like a problem of GItHub in general.

marianfoo commented 2 years ago

I might have tested too much. Is it possible to add a wait time between requests to github? https://docs.github.com/en/rest/overview/resources-in-the-rest-api#secondary-rate-limits https://docs.github.com/en/rest/guides/best-practices-for-integrators#dealing-with-secondary-rate-limits

IObert commented 2 years ago

Possibly, I'm not sure

marianfoo commented 2 years ago

Hi @IObert i removed the rank from the updated page: https://github.com/SAP-samples/artifact-of-the-month/pull/14/commits/2f189e4463ed91a241311257837afe09829e9bd8 https://github.com/SAP-samples/artifact-of-the-month/pull/14/commits/a856a7367245692ffefee17b8d868d4d31e535c9

implemented a rate limit throttle on octokit https://github.com/SAP-samples/artifact-of-the-month/pull/14/commits/a6e8c8e04f7ef37582d35c2ea7eb448c30d15c30 I used this template https://octokit.github.io/rest.js/v18#throttling

The new action ran succesfully Idid not run in rate limit issues, apperently the limit on authenticated requests is 5.000 per hours, which should be enough: https://github.com/marianfoo/artifact-of-the-month/runs/4301373312?check_suite_focus=true

The changes are visible on my github pages site: https://marianfoo.github.io/artifact-of-the-month/#/updated

IObert commented 2 years ago

Awesome! Many thanks 🥳