google / mtail

extract internal monitoring data from application logs for collection in a timeseries database
Apache License 2.0
3.82k stars 378 forks source link

fix: Return to read immediately after a successful read. #894

Closed jaqx0r closed 2 months ago

jaqx0r commented 2 months ago

Copying the filestream, now pipe, dgram, and socket streams return to read again immediately after a successful read, so that we don't wait.

This is now obviously the problem in #685 and using the bandwidth-delay-product we can see that a 250ms pause between reads of 4096B and 128KiB matches the results seen.

Before:

jaq% time ./mtail -logs - -progs examples/rsyncd.mtail < internal/mtail/testdata/rsyncd.log
0.01s user 0.01s system 7% cpu 0.264 total

After:

jaq% time ./mtail -logs - -progs examples/rsyncd.mtail < internal/mtail/testdata/rsyncd.log
0.01s user 0.03s system 102% cpu 0.041 total

Thanks to @rideliner for the hint.

Issue: #685

github-actions[bot] commented 2 months ago

Unit Test Results

    1 files     27 suites   8m 40s :stopwatch:   648 tests   646 :white_check_mark: 1 :zzz: 1 :x: 1 917 runs  1 913 :white_check_mark: 3 :zzz: 1 :x:

For more details on these failures, see this check.

Results for commit afd77e48.

:recycle: This comment has been updated with latest results.