search

zendesk / ultragrep

the grep that greps the hardest.
Apache License 2.0
28 stars 4 forks source link

Voice logs not being searched correctly #29

Open ciaranarcher opened 10 years ago

ciaranarcher commented 10 years ago

We're trying to use ultragrep to search voice logs. @mphoratiu helped us actually search the voice logs, but it's still not right.

Here's an example, we're trying to search for the Twilio timeout string on pod2 (logs1):

First, lets prove we can find the string using regular grep:

Now we can try it with ultragrep:

This thing to note is that search date is never increasing.

It does say that it is searching the right log file however, e.g.

Moreover, results are output but they are not the log lines we're looking for, they just seem to be a sample of random voice logs :anguished: - [here is a sample]() with no sign of our Twilio timeout search string.

My guess is something is wrong with the voice configuration. Can someone give us a hand with this please? Thanks in advance!

mphoratiu commented 10 years ago

This is related.. https://github.com/zendesk/ultragrep_deploy/pull/5

osheroff commented 10 years ago

your logs are all fakata:

Started GET "/api/v2/channels/voice/stats/agents_activity.json?page=1087&per_page=1" for 209.190.73.37 at 2014-04-23 23:59:52 +0000
Started GET "/api/v2/channels/voice/stats/agents_activity.json?page=1087&per_page=1" for 209.190.73.37 at 2014-04-23 23:59:52 +0000
Started GET "/api/v2/channels/voice/stats/agents_activity.json?page=555&per_page=1" for 209.190.73.37 at 2014-04-23 23:59:52 +0000
Started GET "/api/v2/channels/voice/stats/agents_activity.json?page=555&per_page=1" for 209.190.73.37 at 2014-04-23 23:59:52 +0000
Processing by Api::V2::StatsController#agents_activity as JSON
Processing by Api::V2::StatsController#agents_activity as JSON
  Parameters: {"page"=>"555", "per_page"=>"1"}
  Parameters: {"page"=>"555", "per_page"=>"1"}
api/v2/stats/agents_activity does not have any allowed parameters
api/v2/stats/agents_activity does not have any allowed parameters
Processing by Api::V2::StatsController#agents_activity as JSON
Processing by Api::V2::StatsController#agents_activity as JSON
  Parameters: {"page"=>"591", "per_page"=>"1"}
  Parameters: {"page"=>"591", "per_page"=>"1"}
api/v2/stats/agents_activity does not have any allowed parameters
api/v2/stats/agents_activity does not have any allowed parameters
Processing by Api::V2::StatsController#agents_activity as JSON
Processing by Api::V2::StatsController#agents_activity as JSON
  Parameters: {"page"=>"1087", "per_page"=>"1"}
  Parameters: {"page"=>"1087", "per_page"=>"1"}
api/v2/stats/agents_activity does not have any allowed parameters
api/v2/stats/agents_activity does not have any allowed parameters

notice how all the requests are interleaved together, meaning ultragrep can't make heads or tails of them.

Please use ActiveSupport::BufferedLogger or similar to de-interleave the logs, then ultragrep will probably be able to read them.

Note also that ultragrep requires two newlines between log entries (at least for the "app" format).