Open Slach opened 1 year ago
@Slach could you try this?
func (ch *ClickHouse) LogQuery(query string, args ...interface{}) string {
level := zerolog.InfoLevel
if !ch.Config.LogSQLQueries {
level = zerolog.DebugLevel
}
if len(args) > 0 {
ch.Logger.WithLevel(level).Msg(strings.NewReplacer("\n", " ", "\r", " ", "\t", " ").Replace(fmt.Sprintf("%s with args %v", query, args)))
} else {
ch.Logger.WithLevel(level).Msg(strings.NewReplacer("\n", " ", "\r", " ", "\t", " ").Replace(query))
}
return query
}
Trying to confirm if that might help.
@sruehl I can't thank you enough, it works, you save many debug hours
=( but found race condition in other place
WARNING: DATA RACE
Write at 0x00c000448655 by goroutine 34:
github.com/rs/zerolog/internal/json.Encoder.AppendString()
/home/runner/go/pkg/mod/github.com/rs/zerolog@v1.30.0/internal/json/string.go:63 +0x71d
github.com/rs/zerolog/internal/json.Encoder.AppendKey()
/home/runner/go/pkg/mod/github.com/rs/zerolog@v1.30.0/internal/json/base.go:18 +0x275
github.com/rs/zerolog.(*Event).Str()
/home/runner/go/pkg/mod/github.com/rs/zerolog@v1.30.0/event.go:256 +0xea
github.com/Altinity/clickhouse-backup/pkg/backup.(*Backuper).downloadTableMetadata()
/home/runner/work/clickhouse-backup/clickhouse-backup/pkg/backup/download.go:412 +0x3c24
github.com/Altinity/clickhouse-backup/pkg/backup.(*Backuper).Download.func2()
/home/runner/work/clickhouse-backup/clickhouse-backup/pkg/backup/download.go:197 +0x44e
golang.org/x/sync/errgroup.(*Group).Go.func1()
/home/runner/go/pkg/mod/golang.org/x/sync@v0.3.0/errgroup/errgroup.go:75 +0x133
Previous write at 0x00c000448655 by goroutine 33:
github.com/rs/zerolog/internal/json.Encoder.AppendString()
/home/runner/go/pkg/mod/github.com/rs/zerolog@v1.30.0/internal/json/string.go:63 +0x71d
github.com/rs/zerolog/internal/json.Encoder.AppendKey()
/home/runner/go/pkg/mod/github.com/rs/zerolog@v1.30.0/internal/json/base.go:18 +0x275
github.com/rs/zerolog.(*Event).Str()
/home/runner/go/pkg/mod/github.com/rs/zerolog@v1.30.0/event.go:256 +0xea
github.com/Altinity/clickhouse-backup/pkg/backup.(*Backuper).downloadTableMetadata()
/home/runner/work/clickhouse-backup/clickhouse-backup/pkg/backup/download.go:412 +0x3c24
github.com/Altinity/clickhouse-backup/pkg/backup.(*Backuper).Download.func2()
/home/runner/work/clickhouse-backup/clickhouse-backup/pkg/backup/download.go:197 +0x44e
golang.org/x/sync/errgroup.(*Group).Go.func1()
/home/runner/go/pkg/mod/golang.org/x/sync@v0.3.0/errgroup/errgroup.go:75 +0x133
backup/download.go:412
// simple import zerolog
import (
"github.com/rs/zerolog/log"
)
func (b *Backuper) downloadTableMetadata(ctx context.Context, backupName string, disks []clickhouse.Disk, tableTitle metadata.TableTitle, schemaOnly bool, partitions []string) (*metadata.TableMetadata, uint64, error) {
...
log.Info().
Str("operation", "download_metadata").
Str("backup", backupName).
Str("duration", utils.HumanizeDuration(time.Since(start))).
Str("size", utils.FormatBytes(size)).
Msg("done")
return &tableMetadata, size, nil
}
just one function without instances of zerolog.Logger, which called inside pool of go-routines
downloadSemaphore := semaphore.NewWeighted(int64(b.cfg.General.DownloadConcurrency))
metadataGroup, metadataCtx := errgroup.WithContext(ctx)
for i, t := range tablesForDownload {
...
metadataGroup.Go(func() error {
defer downloadSemaphore.Release(1)
if err != nil {
return err
}
})
}
if err := metadataGroup.Wait(); err != nil {
return fmt.Errorf("one of Download Metadata go-routine return error: %v", err)
}
Any ideas why it happen? and how to resolve it?
main.go with zerolog initialization
import (
stdlog "log"
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
"github.com/rs/zerolog/pkgerrors"
)
func main() {
zerolog.TimeFieldFormat = zerolog.TimeFormatUnixMs
zerolog.ErrorStackMarshaler = pkgerrors.MarshalStack
consoleWriter := zerolog.ConsoleWriter{Out: os.Stdout, NoColor: true, TimeFormat: "2006-01-02 15:04:05.000"}
log.Logger = zerolog.New(zerolog.SyncWriter(consoleWriter)).With().Timestamp().Caller().Logger()
stdlog.SetOutput(log.Logger)
...
}
@rs i have race-condition ;( https://github.com/Altinity/clickhouse-backup/actions/runs/5259410860/jobs/9504979925#step:7:2222
when try to use zerolog with multiple go-routines + errorgroup
LogQuery() related code
How race condition possible here? i redefine
log.Logger
once in main functionand after it use copy of logger
Logger: log.With().Str("logger", "clickhouse").Logger(),
and copy of copylogger := ch.Logger.With().Logger()
during execution of With() it shall create copy of
Logger
what i'm doing wrong?