[question] How to stream remote logs?

mbok / logsniffer

logsniffer is a sophisticated open source web tool for parsing, viewing, monitoring and analyzing log data - smarter, collaborative and easier. [No longer maintaned]

GNU Lesser General Public License v3.0

104 stars 46 forks source link

[question] How to stream remote logs? #91

Open archenroot opened 7 years ago

archenroot commented 7 years ago

Hi community,

I would like to tail remote files. Now, I can implement SSH based wrapper for log source, but not sure about the performance especially with increasing number of files to be captured...by adopting: https://github.com/lucastheisen/jsch-nio

Another idea is to implement agent based stuff via https://github.com/OpenHFT/Chronicle-Queue.

Please any comments or better proposals are welcomed.

archenroot commented 7 years ago

Ok, I decided to simplify things and under the terms that I don't have hundreds of servers to monitor, I will go with SSHFS.

SSHFS could be quite slow, so for best performance I will:

patch all servers with https://www.psc.edu/hpn-ssh.
mounting with options Ciphers=arcfour(fastest cipher) -o Compression=no (no compression)

mbok commented 7 years ago

Looking forward for your submission