[Bug]: run-eval workflow is failing on git pull (due to fork?)

neubig commented 2 days ago

Is there an existing issue for the same bug?

[X] I have checked the existing issues.

Describe the bug and reproduction steps

I tried to run the eval workflow on this PR (from a fork): https://github.com/All-Hands-AI/OpenHands/pull/5220

It is failing with the following error:

Fetching the repository
  /usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +refs/heads/add_thinking*:refs/remotes/origin/add_thinking* +refs/tags/add_thinking*:refs/tags/add_thinking*
  The process '/usr/bin/git' failed with exit code 1
  Waiting 19 seconds before trying again
  /usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +refs/heads/add_thinking*:refs/remotes/origin/add_thinking* +refs/tags/add_thinking*:refs/tags/add_thinking*
  The process '/usr/bin/git' failed with exit code 1
  Waiting 16 seconds before trying again
  /usr/bin/git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +refs/heads/add_thinking*:refs/remotes/origin/add_thinking* +refs/tags/add_thinking*:refs/tags/add_thinking*

I suspect that this might be due to the eval workflow not working on forks. @mamoodi it'd be great if you can investigate!

OpenHands Installation

Other

OpenHands Version

No response

Operating System

None

Logs, Errors, Screenshots, and Additional Context

No response

mamoodi commented 2 hours ago

To comment, I'm aware of the issue. The fix needs a little pondering.