sum.golang.org: repeated 500 internal server errors when fetching very new versions

mvdan commented 1 month ago

We have had four CI jobs fail in the past week due to sum.golang.org reporting 500 internal server errors as well as an unexpected EOF.

The two types of errors we have seen look like:

go: cuelang.org/go@v0.11.0-alpha.3.0.20241018144220-537f744a9cc7: verifying go.mod: cuelang.org/go@v0.11.0-alpha.3.0.20241018144220-537f744a9cc7/go.mod: reading https://sum.golang.org/lookup/cuelang.org/go@v0.11.0-alpha.3.0.20241018144220-537f744a9cc7: 500 Internal Server Error

go: cuelang.org/go@v0.10.1: verifying module: cuelang.org/go@v0.10.1: Get "https://sum.golang.org/lookup/cuelang.org/go@v0.10.1": EOF

These CI jobs are doing what is effectively:

go mod init
go get cuelang.org/go@${commitRef}

where ${commitRef} is a commit hash which was just pushed to master.

We are aware that proxy.golang.org and sum.golang.org may respond with status codes like 404 for the first fifteen to thirty minutes as they discover the new version (https://github.com/golang/go/issues/49916#issuecomment-984839295), but I still assume that the 500 and EOF errors are unexpected and should be fixed.

Moreover, we do dozens of these runs per week, and only very few fail, so sum.golang.org often answers these requests correctly without any issue.

gabyhelp commented 1 month ago

Related Issues and Documentation

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

cagedmantis commented 1 month ago

cc @golang/tools-team

ansaba commented 3 weeks ago

cc: @samthanawalla

samthanawalla commented 3 weeks ago

I'll look into this more after the freeze but it makes sense to return 404 instead of 500.

I suspect it is due to timing out while waiting for the fetch to finish so we should return 404 early in that case and not wait.

golang / go

sum.golang.org: repeated 500 internal server errors when fetching very new versions #70042