tstack / lnav

Log file navigator
http://lnav.org
BSD 2-Clause "Simplified" License
7.99k stars 322 forks source link

Lnav seems to stop processing new data when piped from another source #1118

Open nikess opened 1 year ago

nikess commented 1 year ago

version v0.11.1

Describe the bug I'm piping log data from a separate command. For reference, the command is

nomad alloc logs -tail -f $job_alloc  | lnav -q

(Nomad is https://www.nomadproject.io/ - and that command gets logs from a running remote process. )

When I run that command alone, but NOT piped to lnav, it runs forever. Realtime logs.

When I run it piped to lnav as per above, it runs for about 15 minutes, then lnav just stops. No crashes.

To Reproduce See above.

I appreciate that this involves another tool, but I'm confident nomad is not intrinsically at fault, per above.

If I instead save a (very large) log file from nomad to disk first, and cat that to lnav, it all works.

Is there a good way to debug this? I tried adding the -d , but it's output yields nothing interesting - the last line is just a notification about adding new lines.

Thank you.

otonoton commented 4 months ago

I have the same problem. It seems what is happening for me at least, is that lnav keeps starting a new "file" for every 10MB worth of logs, and the display doesn't automatically move to the newest file, so you have to press Tab to go to Files and then select the last entry at the bottom with Enter, and now you can keep viewing the logs (until it happens again). Very annoying.

tstack commented 4 months ago

@nikess The implementation of handling data piped into lnav was changed in v0.12.0 . Can you try upgrading and seeing if there is any improvement?

@otonoton What version are you using?

detj commented 2 months ago

I'm facing similar issue when piping docker compose logs and after applying a filter.

docker compose logs -f my-service | lnav

Applying a filter with :filter-out foobar. The filter works, but then new log lines are not appended.