pinojs / pino-elasticsearch

🌲 load pino logs into Elasticsearch
MIT License
179 stars 67 forks source link

system and pino-elasticsearch #18

Closed alexgoldstone closed 6 years ago

alexgoldstone commented 6 years ago

I am trying to launch my application via systems (see the following article for details:

https://blog.codeship.com/running-node-js-linux-systemd/

Any suggestions how this might work with pino-elasticsearch. Simply piping does not seem to work.

mcollina commented 6 years ago

This looks more like something related to pm2.

http://pm2.keymetrics.io/docs/usage/log-management/

I'm not a user, but maybe we can figure it out.

@jsumners what would you recommend at this point?

jsumners commented 6 years ago

@mcollina sounds like he wants to use systemd and not pm2.

@alexgoldstone it depends on how new your systemd is. The version that ships with RHEL7/Centos7 does not support this, but newer versions have an option that can be passed to a unit file's StandardOut= to write the logs directly to a file.

So, one could do StandardOut=file:/var/log/myapp.log. Then you would use logagent to ship the logs off to whatever service you like.

alexgoldstone commented 6 years ago

Thanks @jsumners

I am using Ubuntu 16.04 LTS which does not seem to have the StandardOut option but it sounds like that will be the way to go in the future. Do you have any short-term suggestions for older systems?

jsumners commented 6 years ago

Unfortunately systemd is garbage. So not really. But maybe you could cook something up with journalctl -o cat --no-pager -fu your_service.

alexgoldstone commented 6 years ago

For anyone looking for a similar solution once this issue is archived, the following seems to work:

ExecStart=/bin/sh -c "/usr/bin/node myApp.js | pino-elasticsearch --host https://elastic-search-service --port 443 --index myIndex"

I was having trouble getting my environment variables to be picked-up when I tried this earlier but must not have had quite the same as the above.

@jsumners This is off topic but since you clearly have more experience, I opted to use systems as I had read some negatives re pm2. Based on what you said re systems, would you opt for pm2 or something else given the option?

jsumners commented 6 years ago

No. I vehemently dislike systemd, but we are stuck with it and pm2 is cumbersome to use. I don't see any reason to use pm2 unless you are all-in on their environment (pay for their APM service).

That being said, I have recently been considering using https://github.com/immortal/immortal for some services. It follows the runit style of process management and makes it easy to provide user-level service management. But I haven't had time to fully vet it for production use.

alexgoldstone commented 6 years ago

Thanks. I'll check it out. In the interim I have been able to achieve what I set out to with a combination of systemd and this pino-elasticsearch project so thank you again. I will close this ticket but my solution is noted above if anyone else searches.