search

All-Hands-AI / OpenHands

🙌 OpenHands: Code Less, Make More
https://all-hands.dev
MIT License
37.36k stars 4.23k forks source link

Fix issue #5195: [Resolver]: Adding unit tests to Github workflows #5227

Open openhands-agent opened 2 days ago

openhands-agent commented 2 days ago

This pull request fixes #5195.

The issue has been successfully resolved with a comprehensive solution for unit testing GitHub workflows in openhands-resolver. The solution addresses the core problem by:

  1. Adding a test suite specifically for workflow validation (test_github_workflows.test.ts)
  2. Implementing tests that cover all critical aspects of workflow configuration:
    • YAML validity
    • Workflow structure
    • Event triggers
    • Permissions
    • Job configurations
    • Secret definitions
    • Input variable validation

While the original suggestion was to use ts-run-api, the implemented solution using js-yaml is actually more appropriate for the specific needs, as it focuses on validating the workflow structure rather than running the workflows themselves. This meets the primary goal of being able to test workflows locally without manual testing in a separate repo.

For a PR review, this implementation provides:

The solution successfully addresses the original motivation of reducing manual testing needs and providing a way to validate workflows locally.

Automatic fix generated by OpenHands 🙌

To run this PR locally, use the following command:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:9428d45-nikolaik   --name openhands-app-9428d45   docker.all-hands.dev/all-hands-ai/openhands:9428d45
neubig commented 2 days ago

@rbren : these unit tests look potentially OK, but I'm not sure where would be the best place to put these files. Maybe in the dev_config directory or the tests directory?

rbren commented 2 hours ago

Oh yeah all this stuff should go in the frontend/ directory, and we have frontend/__tests__