golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
120.96k stars 17.36k forks source link

x/build: gotip-netbsd-arm64 builder stops working #66963

Open cherrymui opened 2 months ago

cherrymui commented 2 months ago

https://ci.chromium.org/ui/p/golang/builders/luci.golang.ci/gotip-netbsd-arm64

Looks like the builder is idle, but all builds before failed with infra failure.

cc @golang/netbsd

bsiegert commented 2 months ago

I run this bot.

It failed a build with BOT_DIED on April 19, and I rebooted it on April 21. It is now doing other tasks but it looks like tasks that came up while the bot was down are not automatically retried.

I asked on internal chat if I can have a notification for BOT_DIED tasks. Unfortunately, the storage in this Raspberry Pi (a USB HDD) is a bit flaky and occasionally goes into "Input/output error" mode.

bsiegert commented 2 months ago

By the way, the bot wasn't idle, it ran tests on /x/ repositories.

cherrymui commented 2 months ago

By the way, the bot wasn't idle, it ran tests on /x/ repositories.

https://ci.chromium.org/ui/p/golang/builders/luci.golang.ci/gotip-netbsd-arm64 this page showed it was idle when I looked this morning. Perhaps there was no commit to build at the time. I mostly meant to say it was not "Quarantined" or "Offline".

Thanks for resetting the builder. It seems to work well now. Let's call it fixed.

cherrymui commented 1 month ago

It seems the builder stopped working again, since Apr 24. https://ci.chromium.org/ui/p/golang/builders/luci.golang.ci/gotip-netbsd-arm64 Thanks.

dmitshur commented 1 month ago

It's currently online and working, so closing. Thanks.

bsiegert commented 1 month ago

It's offline again.

After a reboot, it stayed up only for a couple of hours before crashing. The last time this happened, I had a console connected and could see errors from the disk, so it looks like the hard drive is dying. I will swap the disk but I can only do that in ~1 week once I am back from my business trip.