Open LoopedBard3 opened 4 months ago
Machines have been removed: https://github.com/dotnet/dnceng/issues/2596
Seems other machines may now be hitting the same issue.
Updating this issue to track updating how we install the lttng version to get around the bug.
The following bug caused us to remove the lttng-modules-dkms package for every Linux microbenchmark run: https://bugs.launchpad.net/ubuntu/+source/lttng-modules/+bug/2043004, thanks @caaavik-msft. This issue tracks updating how we install the lttng-modules to workaround this problem longterm. It seems that manually installing latest lttng rather than from the Ubuntu package repository may fix the issue.
Here is a sample of the original error:
Looking at job 9e113869-98a8-4c88-893c-9433b5a33072, the only two machines that seem to be affected are PERFTIGER138 and 139.
Digging into the Azure Data Explorer, the failure rate of those two machines is >97% while all others are much lower. I messaged to get these machines taken offline for investigation. Looking at the daily failure rates, it seems that the failure started happening consistently on the 12. We will want to rerun the runs between the 12 and 15th.