jeanphorn / log4go

a logging package for golang similar to log4j or log4c++ supporting console, file and network.
137 stars 75 forks source link

log4go block the caller when write to file failed #22

Open FateTHarlaown opened 4 years ago

FateTHarlaown commented 4 years ago

our service uses log4go,we found it will block our program when it got a error because writing file failed. For example, when disk was full, we got an error "FileLogWriter("../log/syncer.log"): write ../log/syncer.log: no space left on device" in stderr and our code was blocked when it try to log. the stack like this

         1   runtime.gopark
             runtime.goparkunlock
             runtime.chansend
             runtime.chansend1
             github.com/jeanphorn/log4go.(*FileLogWriter).LogWrite
             github.com/jeanphorn/log4go.Logger.intLogf
             github.com/jeanphorn/log4go.Info
             teledb/pkg/syncer.(*Controller).coordinator
             teledb/pkg/syncer.(*Controller).Run
             main.main.func2

maybe that is because the background goroutine will return when it meet a error

go func() {
        defer func() {
            if w.file != nil {
                fmt.Fprint(w.file, FormatLogRecord(w.trailer, &LogRecord{Created: time.Now()}))
                w.file.Close()
            }
        }()

        for {
            select {
            case <-w.rot:
                if err := w.intRotate(); err != nil {
                    fmt.Fprintf(os.Stderr, "FileLogWriter(%q): %s\n", w.filename, err)
                    return
                }
            case rec, ok := <-w.rec:
                if !ok {
                    return
                }
                now := time.Now()
                if (w.maxlines > 0 && w.maxlines_curlines >= w.maxlines) ||
                    (w.maxsize > 0 && w.maxsize_cursize >= w.maxsize) ||
                    (w.daily && now.Day() != w.daily_opendate) {
                    if err := w.intRotate(); err != nil {
                        fmt.Fprintf(os.Stderr, "FileLogWriter(%q): %s\n", w.filename, err)
                        return
                    }
                }

                // Sanitize newlines
                if w.sanitize {
                    rec.Message = strings.Replace(rec.Message, "\n", "\\n", -1)
                }

                // Perform the write
                n, err := fmt.Fprint(w.file, FormatLogRecord(w.format, rec))
                if err != nil {
                    fmt.Fprintf(os.Stderr, "FileLogWriter(%q): %s\n", w.filename, err)
                    return
                }

                // Update the counts
                w.maxlines_curlines++
                w.maxsize_cursize += n
            }
        }
    }()

and then the channel w.rec will soon be full, and block the writer

// This is the FileLogWriter's output method
func (w *FileLogWriter) LogWrite(rec *LogRecord) {
    w.rec <- rec
}

can we find a better way that will not make our service block ?

amao5466 commented 4 years ago

I met the same trouble with you

jlulxy commented 2 years ago

I met the same trouble with you