[Error: EAGAIN: resource temporarily unavailable, scandir 'MyNFSMOUNTPOINT']

[jerome83136]

Hello,

I have had this error in my logs after I have restarted my autofs service.

It seems node-logstash stopped reading files exposed from a NFS mountpoint and is unable to resume reading them when the mountpoint becomes available again.

[Wed, 22 Jun 2016 15:39:19 GMT] ERROR Unable to read directory /MyNFSMOUNTPOINT { [Error: EAGAIN: resource temporarily unavailable, scandir '/MyNFSMOUNTPOINT']
  errno: -11,
  code: 'EAGAIN',
  syscall: 'scandir',
  path: '/MyNFSMOUNTPOINT' }
[Wed, 22 Jun 2016 15:39:19 GMT] ERROR [input_file] Error: EAGAIN: resource temporarily unavailable, scandir '/MyNFSMOUNTPOINT'
[Wed, 22 Jun 2016 15:39:19 GMT] ERROR Unable to read directory /MyNFSMOUNTPOINT { [Error: EAGAIN: resource temporarily unavailable, scandir '/MyNFSMOUNTPOINT']
  errno: -11,
  code: 'EAGAIN',
  syscall: 'scandir',
  path: '/MyNFSMOUNTPOINT' }

Can someone help me to figure this out ? Jérôme

[bpaquet]

Hi,

To be honest, I'm not able to read files from NFS. It's not a surprise, because node-logstash use Node fs.watch api, which use libinotify, which does not work on NFS. So, can you detail your config which seems to work on NFS ?

Another point : why do you read files over NFS ? Node logstash and ELK stack are designed, from my point of view, to avoid that. You have lot of way to transfer files between servers without using NFS : standard syslog, and all Node-Logstash / Logstash transports : AMQP, Redis, ZeroMQ, TCP, Lumberjack... Can you detail what are you trying to do ?

[jerome83136]

Hi, I'm not really using NFS, I said this to keep things simple. I'm using SSHFS to centralize all my webservers (Apache) logs on a single server which I use to work on my logs. I don't now if SSHFS is libnotify compliant or not, thought.

I didn't want to implement things that way, but my security team let me no choice :(

My idea was first to install node-logstash on my webservers (DMZ) and to use active-mq or anything else to send events to a node-logstash agent (LAN) which would then send events to Elasticsearch.

But it was rejected for security purpose (which I don't agree with), and I had to find another solution to centralize my logs. The security team only allowed me to make connections from the LAN to the DMZ, not the opposite.

I know that implementation is uncommon and certainly not recommended, but I had no choice :(

If using NFS or SSHFS is not supported, I can understand you don't want to loose more time with such a strange setup. Just let me now and I will close the issue (you already helped a lot)

I have opened it because it was working with the "standard" logstash, but I guess they are not working the same way.

Best regards Jérôme