[Bug]: Github resolver looking for wrong PR number

neubig commented 3 hours ago

Is there an existing issue for the same bug?

[X] I have checked the existing issues.

Describe the bug and reproduction steps

When responding to an issue on this PR: https://github.com/All-Hands-AI/OpenHands/pull/5058

The github resolver is getting an error like this:

ERROR:root:<class 'requests.exceptions.HTTPError'>: 404 Client Error: Not Found for url: https://api.github.com/repos/All-Hands-AI/OpenHands/issues/12809

I'm not sure why it's looking for issue 12809 despite the fact that the PR is 5058. We should investigate openhands-resolver.yml and any relevant files in the openhands/resolver directory to find the error, fix it, and write a test to make sure it doesn't happen again.

OpenHands Installation

Docker command in README

OpenHands Version

No response

Operating System

None

Logs, Errors, Screenshots, and Additional Context

Here is the full trace:

Run cd /tmp && python -m openhands.resolver.resolve_issue \
  cd /tmp && python -m openhands.resolver.resolve_issue \
    --repo All-Hands-AI/OpenHands \
    --issue-number 5058 \
    --issue-type pr \
    --max-iterations 50 \
    --comment-id None
  shell: /usr/bin/bash -e {0}
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.1[2](https://github.com/All-Hands-AI/OpenHands/actions/runs/11861706839/job/33059571872?pr=5058#step:10:2).7/x64
    PKG_CONFIG_PATH: /opt/hostedtoolcache/Python/[3](https://github.com/All-Hands-AI/OpenHands/actions/runs/11861706839/job/33059571872?pr=5058#step:10:3).12.7/x64/lib/pkgconfig
    Python_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x6[4](https://github.com/All-Hands-AI/OpenHands/actions/runs/11861706839/job/33059571872?pr=5058#step:10:4)
    Python2_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    Python3_ROOT_DIR: /opt/hostedtoolcache/Python/3.12.7/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.12.7/x64/lib
    ISSUE_NUMBER: [5](https://github.com/All-Hands-AI/OpenHands/actions/runs/11861706839/job/33059571872?pr=5058#step:10:5)058
    ISSUE_TYPE: pr
    COMMENT_ID: None
    MAX_ITERATIONS: 50
    SANDBOX_ENV_GITHUB_TOKEN: ***
    GITHUB_TOKEN: ***
    GITHUB_USERNAME: ***
    LLM_MODEL: ***
    LLM_API_KEY: ***
    LLM_BASE_URL: ***
    PYTHONPATH: 
ERROR:root:  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/opt/hostedtoolcache/Python/3.12.7/x[6](https://github.com/All-Hands-AI/OpenHands/actions/runs/11861706839/job/33059571872?pr=5058#step:10:6)4/lib/python3.12/site-packages/openhands/resolver/resolve_issue.py", line 624, in <module>
    main()
  File "/opt/hostedtoolcache/Python/3.12.[7](https://github.com/All-Hands-AI/OpenHands/actions/runs/11861706839/job/33059571872?pr=5058#step:10:7)/x64/lib/python3.12/site-packages/openhands/resolver/resolve_issue.py", line 604, in main
    asyncio.run(
  File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/asyncio/runners.py", line 11[8](https://github.com/All-Hands-AI/OpenHands/actions/runs/11861706839/job/33059571872?pr=5058#step:10:8), in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/openhands/resolver/resolve_issue.py", line 341, in resolve_issue
    issues: list[GithubIssue] = issue_handler.get_converted_issues(
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/openhands/resolver/issue_definitions.py", line 4[9](https://github.com/All-Hands-AI/OpenHands/actions/runs/11861706839/job/33059571872?pr=5058#step:10:9)7, in get_converted_issues
    closing_issues = self.__get_context_from_external_issues_references(
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/openhands/resolver/issue_definitions.py", line 464, in __get_context_from_external_issues_references
    response.raise_for_status()
  File "/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages/requests/models.py", line [10](https://github.com/All-Hands-AI/OpenHands/actions/runs/11861706839/job/33059571872?pr=5058#step:10:10)24, in raise_for_status
    raise HTTPError(http_error_msg, response=self)

ERROR:root:<class 'requests.exceptions.HTTPError'>: 404 Client Error: Not Found for url: https://api.github.com/repos/All-Hands-AI/OpenHands/issues/[12](https://github.com/All-Hands-AI/OpenHands/actions/runs/11861706839/job/33059571872?pr=5058#step:10:12)809
Error: Process completed with exit code 1.

github-actions[bot] commented 3 hours ago

OpenHands started fixing the issue! You can monitor the progress here.

github-actions[bot] commented 3 hours ago

A potential fix has been generated and a draft PR #5062 has been created. Please review the changes.

All-Hands-AI / OpenHands