search

hpcloud / tail

Go package for reading from continously updated files (tail -f)
MIT License
2.71k stars 503 forks source link

Solved the problem for never return the last line if it's not followed by a newline #126

Open Code-Hex opened 7 years ago

Code-Hex commented 7 years ago

What was the problem?

Even though enable the Follow mode, Could not get the last line if it is not newline on end of the line.

For example, I had read the file that is not followed by a newline on the last line using github.com/hpcloud/tail. file is like this.

host:192.168.12.34  user:-  time:2015-09-06T05:58:05+09:00  method:POST uri:/foo/bar?token=xxx&uuid=1234    protocol:HTTP/1.1   status:200  size:12 api_status:-    request_time:0.247  apptime:0.057   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T05:58:41+09:00  method:POST uri:/foo/bar?token=yyy  protocol:HTTP/1.1   status:200  size:34 api_status:-    request_time:0.012  apptime:0.100   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:42+09:00  method:GET  uri:/foo/bar?token=zzz  protocol:HTTP/1.1   status:200  size:56 api_status:-    request_time:0.034  apptime:0.123   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:43+09:00  method:GET  uri:/foo/bar    protocol:HTTP/1.1   status:400  size:15 api_status:-    request_time:-  apptime:-   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T05:58:44+09:00  method:POST uri:/foo/bar?token=yyy  protocol:HTTP/1.1   status:200  size:34 api_status:-    request_time:0.123  apptime:0.234   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T05:58:44+09:00  method:POST uri:/hoge/piyo?id=yyy   protocol:HTTP/1.1   status:200  size:34 api_status:-    request_time:0.123  apptime:0.234   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T05:58:05+09:00  method:POST uri:/foo/bar?token=xxx&uuid=1234    protocol:HTTP/1.1   status:200  size:12 api_status:-    request_time:0.247  apptime:0.057   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T05:58:41+09:00  method:POST uri:/foo/bar?token=yyy  protocol:HTTP/1.1   status:200  size:34 api_status:-    request_time:0.012  apptime:0.100   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:42+09:00  method:GET  uri:/foo/bar?token=zzz  protocol:HTTP/1.1   status:200  size:56 api_status:-    request_time:0.034  apptime:0.123   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:43+09:00  method:GET  uri:/foo/bar    protocol:HTTP/1.1   status:400  size:15 api_status:-    request_time:-  apptime:-   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:43+09:00  method:GET  uri:/diary/entry/1234   protocol:HTTP/1.1   status:200  size:15 api_status:-    request_time:-  apptime:0.135   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:43+09:00  method:GET  uri:/diary/entry/5678   protocol:HTTP/1.1   status:200  size:30 api_status:-    request_time:-  apptime:0.432   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:42+09:00  method:GET  uri:/foo/bar?token=zzz  protocol:HTTP/1.1   status:200  size:56 api_status:-    request_time:0.034  apptime:0.123   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:43+09:00  method:GET  uri:/foo/bar    protocol:HTTP/1.1   status:400  size:15 api_status:-    request_time:-  apptime:-   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:43+09:00  method:GET  uri:/diary/entry/1234   protocol:HTTP/1.1   status:200  size:15 api_status:-    request_time:-  apptime:0.135   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:43+09:00  method:GET  uri:/diary/entry/5678   protocol:HTTP/1.1   status:200  size:30 api_status:-    request_time:-  apptime:0.432   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T05:58:05+09:00  method:POST uri:/foo/bar?token=xxx&uuid=1234    protocol:HTTP/1.1   status:200  size:12 api_status:-    request_time:0.247  apptime:0.057   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T05:58:41+09:00  method:POST uri:/foo/bar?token=yyy  protocol:HTTP/1.1   status:200  size:34 api_status:-    request_time:0.012  apptime:0.100   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:42+09:00  method:GET  uri:/foo/bar?token=zzz  protocol:HTTP/1.1   status:200  size:56 api_status:-    request_time:0.034  apptime:0.123   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:43+09:00  method:GET  uri:/foo/bar    protocol:HTTP/1.1   status:400  size:15 api_status:-    request_time:-  apptime:-   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:43+09:00  method:GET  uri:/foo/bar    protocol:HTTP/1.1   status:400  size:15 api_status:-    request_time:-  apptime:-   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2 uuid:-  user_id:-
host:192.168.12.34  user:-  time:2015-09-06T06:00:43+09:00  method:GET  uri:/foo/bar    protocol:HTTP/1.1   status:400  size:15 api_status:-    request_time:-  apptime:-   upstream_addr:127.0.0.1:12345   referer:-   user_agent:curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.4 libidn/1.18 libssh2/1.4.3 uuid:-  user_id:-

But, when I use tail -f, I could get the the last line. We must need to fix this problem.

Thank you.

Code-Hex commented 7 years ago

I think you should not trim newline from the line when write to file. https://github.com/Code-Hex/tail/blob/5ad6447941c464f021ac1c2475ddb04a6e2caf47/tail.go#L221

Because, We can think about these cases. This code compare with tail -f.

for line := range t.Lines {
    fmt.Println(line.Text)
}

I am going to write a some messages.

% printf "hello\n" >> a.txt
% printf wor >> a.txt
% printf "ld\n" >> a.txt

Result,

# tail -f
hello
world(Displayed `ld` after once time displayed `wor`)

# hpcloud/tail
hello
world(It is displayed `world` at once)

# Code-Hex/tail
hello
wor(newlined)
ld

So, If you deleted code to trim the newline, We can get the correct result like follow this.

for line := range t.Lines {
    fmt.Print(line.Text)
}

Then,

# tail -f
hello
world(Displayed `ld` after once time displayed `wor`)

# After remove code to trim newline from the line.
hello
world(Displayed `ld` after once time displayed `wor`)