search

natefinch / lumberjack

lumberjack is a log rolling package for Go
MIT License
4.79k stars 591 forks source link

It doesn´t work in Windows #99

Closed laromicas closed 2 years ago

laromicas commented 4 years ago

With a dockerization of the environment and a Windows Host, writing to the host doesn´t create new files.

hu13 commented 4 years ago

@laromicas is it reproducible for you? I have a windows application using this library and sometimes the log files would no longer be written to until we remove those files and let them recreate for the app to start logging again.

But I cannot come up with a reproducible step.

davidnewhall commented 3 years ago

I add this to my app and pretty much immediately had a window user report a log file that is easily 5x larger than the setting. Same app/settings work fine in Docker/Linux.

natefinch commented 3 years ago

This was written to be cross platform and CI runs the tests on Windows as well as Linux before any change lands. However, that's not to say there couldn't be a bug. If there's a way to reproduce it, I'd be happy to try to figure it out.

davidnewhall commented 3 years ago

So far I've just had the one user report, but my app isn't terribly popular on windows. I will test this myself on a Windows host soon and let you know!

davidnewhall commented 3 years ago

It actually seems not to work on Linux either. The file only rotates when the app starts, so I've probably done something wrong. I'll open a new issue if I can't figure it out. Sorry to bother.

natefinch commented 3 years ago

You have to make sure that all writes to the file go through lumberjack, and that you only have one instance of lumberjack pointed at the same file. The way it works is by looking at the starting size of the file and then keeping track of all writes done to the file, and rotating once you've written starting_size + sum(write_sizes) >= max_size. Outside processes or goroutines writing to the same file won't be tracked by lumberjack and could cause problems.

Basically - you use lumberjack as an io.Writer instead of passing a file to your logger.

davidnewhall commented 3 years ago

That's exactly what I've done. The only thing writing to the file is the logger given to lumberjack. https://github.com/davidnewhall/unpackerr/pull/70/files - I still need to do some more testing on this...... been busy, sorry.

EDIT:

I'm a bit puzzled. I know lumberjack is writing to the log file because I don't pass the log file variable into anything else. I have a logStartInfo() method that prints all my config values, so I know what I'm passing into lumberjack is legit. I saw it rotate the file the first time I started the app with these changes, shown here:

-rw-r--r--    1 root   root    1674177547 Nov 24 22:15 unpackerr-2020-11-24T22-15-32.753.log
-rw-r--r--    1 root   root      88670604 Dec  5 16:52 unpackerr.log
$ du -sch unpackerr*
1.6G    unpackerr-2020-11-24T22-15-32.753.log
101M    unpackerr.log
1.7G    total

But right now it's set to 20 files @ 20 Mb each. I restart the app and it just continues to write to the 101Mb file. Is there something obvious I'm doing wrong here?

EDIT2:

Not sure if it matters, but I'm using a custom *log.Logger on my struct and I only call log.Output to write logs into the Logger. I just can't figure out where my code is failing to initialize this properly...

EDIT 3: I found my problem.

Turns out I was "validating my config" after initializing the logger. That means I was passing zeros in for all my config values and taking the default 100Mb file size. I've fixed it in my code. I found it by adding this line to openExistingorNew

        fmt.Println("lumberjackdebug: ", len(p), l.max(), l.MaxSize)

I suspect this will work for my windows users, and will ask for more feedback. Thank you!

more edit: I'm also in a unique position to log the output errors since I call log.Output() instead of log.Printf()/etc. You can see how I handled that in this PR: https://github.com/davidnewhall/unpackerr/pull/74/files - calling something like this out (in your docs somewhere) may help other users that run into errors.

PS. I have no idea why du says 101M but ls shows more like 90M. 🤷‍♀️

what I have now:

-rw-r--r--    1 root   root    1674177547 Nov 24 22:15 unpackerr-2020-11-24T22-15-32.753.log
-rw-r--r--    1 root   root      90024969 Dec  5 19:44 unpackerr-2020-12-05T19-44-05.358.log
-rw-r--r--    1 root   root          6946 Dec  5 19:44 unpackerr.log
davidnewhall commented 3 years ago

Can confirm this works fine on Windows and every other OS.