battlecode / galaxy

MIT License
10 stars 3 forks source link

Fix infra logging being dropped in certain cases #748

Open n8kim1 opened 7 months ago

n8kim1 commented 7 months ago

Possibly related to #749

In IAP 2024, during the initial deployment setup, things failed. We didn't have logs for this, which was very odd and slowing. Later, we continued to not see logs in some cases.

It's hard to find a concrete cause, so fixing this entirely may be difficult. DWBI if tricky

From Jerry:

meanwhile an issue for infra: why did it fail so badly? there’s an explicit log line wrapping the git clone I suspect the reason is as simple as two things: I think the git package we’re importing is silent-failing when the clone fails. I can’t think of any other explanation to the error. The fmt.Errorf error is only logged to gcloud and we forgot to log.Ctx(ctx).Error().Err(err).Msgf("Error: %v", err) -- there is a log line but I think it logs to the wrong place, ie not to the stream directed toward database

Also:

image