search

anmol098 / waka-readme-stats

This GitHub action helps to add cool dev metrics to your github profile Readme
MIT License
3.14k stars 516 forks source link

Exception: Query 'repo_commit_list' failed to run by returning code of 403: 'message': 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again.' #431

Open SemenMartynov opened 1 year ago

SemenMartynov commented 1 year ago

Describe the bug I find it difficult to describe what caused the bug. Apparently, some requests are executed too often, and it is necessary to add a pause between them.

To Reproduce Just check this log https://github.com/SemenMartynov/SemenMartynov/actions/runs/4421561355/jobs/7772242992

 Traceback (most recent call last):
  File "/waka-readme-stats/main.py", line 221, in <module>
    run(main())
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 208, in main
    stats = await get_stats()
            ^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 156, in get_stats
    yearly_data, commit_data = await calculate_commit_data(repositories)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/yearly_commit_calculator.py", line 37, in calculate_commit_data
    await update_data_with_commit_stats(repo, yearly_data, date_data)
  File "/waka-readme-stats/yearly_commit_calculator.py", line 63, in update_data_with_commit_stats
    commit_data = await DM.get_remote_graphql("repo_commit_list", owner=owner, name=repo_details["name"], branch=branch["name"], id=GHM.USER.node_id)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 293, in get_remote_graphql
    res = await DownloadManager._fetch_graphql_paginated(query, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 267, in _fetch_graphql_paginated
    initial_query_response = await DownloadManager._fetch_graphql_query(query, **kwargs, pagination="first: 100")
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 231, in _fetch_graphql_query
    raise Exception(f"Query '{query}' failed to run by returning code of {res.status_code}: {res.json()}")
Exception: Query 'repo_commit_list' failed to run by returning code of 403: {'documentation_url': 'https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits', 'message': 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again.'}
sys:1: RuntimeWarning: coroutine 'AsyncClient.get' was never awaited
##[debug]Docker Action run completed with exit code 1

Expected behavior To make everything work as it used to.)

Github repository link If applicable, please share the Github repository link.

aravindvnair99 commented 1 year ago

@SemenMartynov it looks like your workflow has been failing for more than month for various reasons. However, the most recent executions are related to the error you're pointing out.

Could you confirm if your GitHub token is unique and not being used elsewhere? Could you regenerate the token and keep it unique to your repository for this workflow?

ssoad commented 1 year ago

I have same issue

Traceback (most recent call last):
  File "/waka-readme-stats/main.py", line 221, in <module>
    run(main())
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 208, in main
    stats = await get_stats()
            ^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 156, in get_stats
    yearly_data, commit_data = await calculate_commit_data(repositories)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/yearly_commit_calculator.py", line 37, in calculate_commit_data
    await update_data_with_commit_stats(repo, yearly_data, date_data)
  File "/waka-readme-stats/yearly_commit_calculator.py", line 63, in update_data_with_commit_stats
    commit_data = await DM.get_remote_graphql("repo_commit_list", owner=owner, name=repo_details["name"], branch=branch["name"], id=GHM.USER.node_id)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line [29](https://github.com/ssoad/ssoad/actions/runs/4432355991/jobs/7780126238#step:3:30)3, in get_remote_graphql
    res = await DownloadManager._fetch_graphql_paginated(query, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 271, in _fetch_graphql_paginated
    query_response = await DownloadManager._fetch_graphql_query(query, **kwargs, pagination=pagination)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 2[31](https://github.com/ssoad/ssoad/actions/runs/4432355991/jobs/7780126238#step:3:32), in _fetch_graphql_query
    raise Exception(f"Query '{query}' failed to run by returning code of {res.status_code}: {res.json()}")
Exception: Query 'repo_commit_list' failed to run by returning code of [40](https://github.com/ssoad/ssoad/actions/runs/4432355991/jobs/7780126238#step:3:41)3: {'documentation_url': 'https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits', 'message': 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again.'}
sys:1: RuntimeWarning: coroutine 'AsyncClient.get' was never awaited

What's the problem ? @anmol098

SemenMartynov commented 1 year ago

The key was definitely unique and was not used anywhere else. But I re-generated it anyway, and in a strange way it turned out to be a solution. https://github.com/SemenMartynov/SemenMartynov/actions/runs/4432343645/jobs/7780279301

@ssoad please try to do the same.

aravindvnair99 commented 1 year ago

Thanks for confirming @SemenMartynov!

I'll keep this open for @ssoad also to confirm.

ssoad commented 1 year ago

@SemenMartynov Thanks. It works @aravindvnair99 Now you can close this issue

aravindvnair99 commented 1 year ago

Thanks for confirming @ssoad!

SemenMartynov commented 1 year ago

@aravindvnair99 failed again =( https://github.com/SemenMartynov/SemenMartynov/actions/runs/4458107908/jobs/7829634589

pseusys commented 1 year ago

Actually before the action waited for 0.3 seconds between the requests. Apparently if we return this wait, the execution time will skyrocket. But we can try if there's no other option.

anmol098 commented 1 year ago

We need to add a interim wait for between the subsequent api calls. Execution time I think should not affect anyone Its like a cron and it should take its own time to complete. For debug run we can probably remove wait time

SemenMartynov commented 1 year ago

Today I see another mistake. I don't know if this is the result of trying to fix the original problem, or if it's some new problem. https://github.com/SemenMartynov/SemenMartynov/actions/runs/4463566749/jobs/7838909571

SemenMartynov commented 1 year ago

Looks good, thx! https://github.com/SemenMartynov/SemenMartynov/actions/runs/4485188763/jobs/7891968341

SemenMartynov commented 1 year ago

Hi guys,

I'm sorry, but I'm getting this error from github again. Perhaps it is worth increasing the pause time, or making it based on Fibonacci numbers or something else... https://github.com/SemenMartynov/SemenMartynov/actions/runs/4642416372/jobs/8216297418

theCapypara commented 1 year ago

Hi! Just wanting to add to this, that I also run into this rate limit for months now: https://github.com/theCapypara/theCapypara/actions/runs/5056669031/jobs/9074422155 There probably needs to be some kind of backoff implemented.

mrepol742 commented 1 year ago

I also have the same problem. Getting the new apikey from wakatime fixed the issue..

mikebronner commented 3 months ago

I'm receiving this as well, getting new API key and Github Token did not help.

Run anmol098/waka-readme-stats@master
/usr/bin/docker run --name wakareadmestatswakareadmestatsmaster_a59d0b --label 16c5e9 --workdir /github/workspace --rm -e "INPUT_WAKATIME_API_KEY" -e "INPUT_GH_TOKEN" -e "INPUT_SHOW_PROJECTS" -e "INPUT_SECTION_NAME" -e "INPUT_PULL_BRANCH_NAME" -e "INPUT_PUSH_BRANCH_NAME" -e "INPUT_SHOW_OS" -e "INPUT_SHOW_EDITORS" -e "INPUT_SHOW_TIMEZONE" -e "INPUT_SHOW_COMMIT" -e "INPUT_SHOW_LANGUAGE" -e "INPUT_SHOW_LINES_OF_CODE" -e "INPUT_SHOW_LANGUAGE_PER_REPO" -e "INPUT_SHOW_LOC_CHART" -e "INPUT_SHOW_DAYS_OF_WEEK" -e "INPUT_SHOW_PROFILE_VIEWS" -e "INPUT_SHOW_SHORT_INFO" -e "INPUT_SHOW_UPDATED_DATE" -e "INPUT_SHOW_TOTAL_CODE_TIME" -e "INPUT_COMMIT_BY_ME" -e "INPUT_COMMIT_MESSAGE" -e "INPUT_COMMIT_USERNAME" -e "INPUT_COMMIT_EMAIL" -e "INPUT_COMMIT_SINGLE" -e "INPUT_LOCALE" -e "INPUT_UPDATED_DATE_FORMAT" -e "INPUT_IGNORED_REPOS" -e "INPUT_SYMBOL_VERSION" -e "INPUT_DEBUG_LOGGING" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_RESULTS_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/mikebronner/mikebronner":"/github/workspace" wakareadmestats/waka-readme-stats:master
Traceback (most recent call last):
  File "/waka-readme-stats/main.py", line 230, in <module>
    run(main())
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 217, in main
    stats = await get_stats()
            ^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/main.py", line 162, in get_stats
    yearly_data, commit_data = await calculate_commit_data(repositories)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/yearly_commit_calculator.py", line 37, in calculate_commit_data
    await update_data_with_commit_stats(repo, yearly_data, date_data)
  File "/waka-readme-stats/yearly_commit_calculator.py", line 63, in update_data_with_commit_stats
    commit_data = await DM.get_remote_graphql("repo_commit_list", owner=owner, name=repo_details["name"], branch=branch["name"], id=GHM.USER.node_id)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 297, in get_remote_graphql
    res = await DownloadManager._fetch_graphql_paginated(query, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 277, in _fetch_graphql_paginated
    query_response = await DownloadManager._fetch_graphql_query(query, **kwargs, pagination=pagination)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 235, in _fetch_graphql_query
    return await DownloadManager._fetch_graphql_query(query, retries_count - 1, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/waka-readme-stats/manager_download.py", line 237, in _fetch_graphql_query
    raise Exception(f"Query '{query}' failed to run by returning code of {res.status_code}: {res.json()}")
Exception: Query 'repo_commit_list' failed to run by returning code of 403: {'documentation_url': 'https://docs.github.com/free-pro-team@latest/rest/overview/rate-limits-for-the-rest-api#about-secondary-rate-limits', 'message': 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again. If you reach out to GitHub Support for help, please include the request ID 1D00:32CE1E:5DE5724:93DC5E8:66084C06.'}
sys:1: RuntimeWarning: coroutine 'AsyncClient.get' was never awaited