Open mkustermann opened 3 years ago
Some of those test failures are fine since they also fail in other existing configurations. (@athomas Do we have a way to diff current results across two configurations?)
Though the crashes we should investigate.
/cc @a-siva
These results are bogus. They were the first attempt to create the builder but it failed because of a missing gen_snapshot. The new builders are baselined on existing ones to avoid this kind of issue, but because the builder didn't work at all everything changed despite the baseline.
This is the builder I'm trying to setup in https://dart-review.googlesource.com/c/sdk/+/204505.
@whesse Do we have some way of removing those results safely?
We can remove the approved failure records, but they will not be replaced by new records representing the diff between the initial baseline and the new configuration. So removing the records, just so these (approved) failures, that are immediately changed to passes, disappear from the feed doesn't seem to gain us much.
We could create result records showing the diff between the initial baseline (from another builder) and the new builder by copying the baseline over the results.json in latest, so the next build will compare itself to the baseline.
Currently, many tests seem to be changing state (randomly?) from crash to pass or compile-time error, and back again, so it may be best to investigate these crashes before trying to get a clean comparison with a baseline.
There are new test failures on [infra] Add more arm64 configurations to the test matrix...[CFE] First pass at internal test utility.
The tests
are failing on configurations