search

shadabahmed / logstasher

Awesome rails logs
MIT License
821 stars 136 forks source link

Occasional empty fields in log entries (ip, route, request_id, hostname, application, environment, system) #107

Closed immitev closed 8 years ago

immitev commented 8 years ago

Hi, sometimes (actually quite often), a log entry is partially empty, i.e. many of its fields are null. This has been observed both in the log file logstash_development.log, and in logstash itself. I am using logstasher (0.9.0) and rails 3.2.22.2. Here is a sample problematic message (notice the "ip" and the rest of the fields are 'null'):

{"method":"GET","path":"/beta/test","format":"html","controller":"beta/test","action":"show","status":404,"duration":57.59,"view":0.71,"db":6.58,"ip":null,"route":null,"request_id":null,"hostname":null,"application":null,"environment":null,"system":null,"source":"logstasher","tags":["request"],"@timestamp":"2016-04-22T11:35:58Z","@version":"1"}

While I see many bad log entries like this, there are also many valid ones, that have all those fields filled properly. Even issuing one and the same request several times in a row (e.g. by clicking a button), leads to sometimes complete and sometimes incomplete log entries. Can't find a particular pattern, though...

Any ideas what could be the reason for this strange behaviour? Any tips for troubleshooting?

immitev commented 8 years ago

After inspecting the code of the gem, it is likely that the method add_default_fields_to_payload in logstasher.rb is not called for some reason, as it is responsible for setting value for ip, route and request_id. But still not clear, what could be the root cause...

highbass commented 8 years ago

i dont even see the ip field in the logs at all... on rails 4

MarcGrimme commented 8 years ago

119 should tackle this. Let me know if you find anything. I also extended the integration test so that those topics will not slip through in future.

edzhelyov commented 8 years ago

@MarcGrimme Yup, custom_params are now logged and issue seems to be fixed in master.

A few important remarks here:

  1. I was forced to fork and allow Rails 3.x version as well, as the current code doesn't work on 3.x. Given the fact that the "0.9.0" has issues and there is nothing that prevents Rails 3.x to run current master. I advice that you lower the gemspec deps back to 3.0.
  2. Release a new version soon.

Thanks. /cc @shadabahmed

shadabahmed commented 8 years ago

I saw some spec failures with 3.x. I can lower the gemspec back and see fix whatever breaks

edzhelyov commented 8 years ago

We have run 0.9 in production for quite a while, so I think it's safe to lower it.