search

getappmap / navie-benchmark

Navie benchmarks
MIT License
0 stars 0 forks source link

feat: Assertion hints #49

Closed kgilpin closed 2 months ago

kgilpin commented 2 months ago

The test generator can get confused about which behavior it's supposed to assert - the bug, or the fix.

So we introduce an extra step that gives it a very direct hint about which way to assert.

kgilpin commented 2 months ago

On comparison of runs 136 vs 271, there's no difference in the success rate of test patch generation.