search

dart-lang / sdk

The Dart SDK, including the VM, dart2js, core libraries, and more.
https://dart.dev
BSD 3-Clause "New" or "Revised" License
9.95k stars 1.53k forks source link

pkg/_macros/test/executor/executor_test is failing #56002

Open a-siva opened 3 weeks ago

a-siva commented 3 weeks ago

The tests

pkg/_macros/test/executor/executor_test RuntimeError (expected Pass)

are failing on configurations

unittest-asserts-release-win-arm64
unittest-asserts-release-win-x64
a-siva commented 3 weeks ago

//cc @jakemac53

dart-github-bot commented 3 weeks ago

area-test, type-bug

The pkg/_macros/test/executor/executor_test tests are failing on Windows ARM64 and x64 configurations, resulting in a RuntimeError instead of the expected Pass.

devoncarew commented 3 weeks ago

around area-test might need to be made more explicit that it should prefer the affected functional area, or otherwise weighted down

I agree with that (and thanks for the feedback on the meta / triaging bot issue). Here, is it area-vm tests that are failing? We don't have a specific area for macros.; we're using area-language for non-implementation discussion issues.

We could improve the prompt for these 'test' cases. We're also not using the tuned model yet, just explicit directions in a prompt; switching to the tuned model should also improve accuracy here.

jakemac53 commented 3 weeks ago

The area should be area-pkg afaik (the tests in question are in pkg/_macros).

jakemac53 commented 3 weeks ago

@a-siva do you have a failure you can link me? I did recently land a fix for this test, and I am not sure how to validate if it was the same error or something different etc.

a-siva commented 3 weeks ago

Here is the first time the failure showed up logs (I believe it was a Timeout initially and then turned into a RuntimeError later).

devoncarew commented 3 weeks ago

The area should be area-pkg afaik (the tests in question are in pkg/_macros).

Yes, but area-pkg is not a great area- itself. It's not triaged - there's no team that does secondary triage on it or ultimately has responsibility for items in that area.

jakemac53 commented 3 weeks ago

Ok yeah looking at the log, that is the one that should be fixed. Is this still failing?

a-siva commented 3 weeks ago

It was failing on the windows bots until I created this issue and approved the failure

jakemac53 commented 1 week ago

Moved this to the "no milestone/release" bucket. It isn't directly tied to any release and I haven't discovered any reasonable way to try and investigate the failure. I intend to just leave this as open tech debt for some time. There are no actual affected users, just a failing test in an environment that I can't reproduce.