bpaquet / node-logstash

Simple logstash implmentation in nodejs : file log collection, sent with zeromq
Other
517 stars 141 forks source link

[stability] Logstash agent don't treat the data for 3 last days #7

Closed abarre closed 12 years ago

abarre commented 12 years ago

For 3 last days, logstash agent don't watch the log file :

See below :

cat /opt/logstash/shared/files.json
{
   "/opt/logstash/shared/log/logstash.log":{
      "index":0
   },
   "/var/log/nginx/access.log":{
      "last_data":"86.35.242.58 - - [23/Oct/2012:20:51:05 +0000] \"GET /w00tw00t.at.ISC.SANS.test0:) HTTP/1.1\" 400 172 \"-\" \"-\"\n",
      "index":33855
   },
   "/var/log/nginx/error.log":{
      "index":0
   },
   "/***api.log":{
      "last_data":"[Tue, 23 Oct 2012 16:16:11 GMT] INFO [memcached] flush of *** : OK\n",
      "index":248294
   },
   "shared/log/production.log":{
      "last_data":"Completed 200 OK in 1ms (Views: 0.4ms | ActiveRecord: 0.0ms)\n",
      "index":5131
   },
   "shared/log/unicorn.stderr.log":{
      "last_data":"I, [2012-10-23T14:51:17.691070 #17644]  INFO -- : worker=2 ready\n",
      "index":1283
   },
   "/shared/log/unicorn.stdout.log":{
      "index":0
   }
}

After restarting logstash

{
   "/opt/logstash/shared/log/logstash.log":{
      "index":0
   },
   "/var/log/nginx/access.log":{
      "last_data":"31.222.158.153 - api-integration.fstrz.net - [26/Oct/2012:12:27:57 +0000] \"PUT ***\"\n",
      "index":121998
   },
   "/var/log/nginx/error.log":{
      "index":0
   },
   "*** api.log":{
      "last_data":"[Fri, 26 Oct 2012 12:27:57 GMT] INFO ",
      "index":763065
   },
   "log/production.log":{
      "last_data":"Completed 200 OK in 1ms (Views: 0.5ms | ActiveRecord: 0.0ms)\n",
      "index":20047
   },
   "unicorn.stderr.log":{
      "last_data":"I, [2012-10-25T13:23:03.363704 #30359]  INFO -- : worker=2 done reopening logs\n",
      "index":314
   },
   "unicorn.stdout.log":{
      "index":0
   }
}
bpaquet commented 12 years ago

If you have restarted node-logstash @ 12h27, it's normal. Files.json is updated only when node-logstash quit. Did you observe some anormal CPU or RAM consumption on node-logstash before restarted it ?