search

vmware / vic

vSphere Integrated Containers Engine is a container runtime for vSphere.
http://vmware.github.io/vic
Other
639 stars 173 forks source link

docker + syslog + supervisor logging problem #8460

Open yg382 opened 5 years ago

yg382 commented 5 years ago

I am trying to send tomcat and apache logs to a centralized syslog. unfortunately, after trying a lot of previous solutions, it didn't work out.

Hint: I am using supervisor to start my services

I tried the following:

1- forwarding all supervisor logs to stdout

2- changed the CATALINA_OUT to stdout and apache error_log to stdout as well but did not work out

3- then I have set the following env in docker-compose as well before build:

SYSLOG_SERVER=ip address
SYSLOG_PORT=514
SYSLOG_PROTO=udp

Note: I am using VMware integrated container (Vic) but I don't think this is the problem as the other containers, not using supervisor, are logging to stdout with no problem

[supervisord]
nodaemon=true
logfile=/dev/stdout
loglevel=debug
logfile_maxbytes=0
[program:apache]
command=/usr/sbin/httpd -D FOREGROUND
autostart=true
autorestart=true
startsecs=5
logfile_maxbytes=0
stdout_logfile_maxbytes=0
stderr_logfile_maxbytes=0
stdout_logfile=/dev/stdout
stderr_logfile=/dev/stdout

[program:tomcat]
command=/opt/cmaas/portal/bin/catalina.sh run
autostart=true
autorestart=true
startsecs=5
logfile_maxbytes=0
stdout_logfile_maxbytes=0
stderr_logfile_maxbytes=0
stdout_logfile=/dev/stdout
stderr_logfile=/dev/stdout

tried this http://mneilsworld.com/discussion/supervisord-docker-and-loggly and installed supervisor-logging and put the correct env variables for the correct syslog server but did not work out

my current supervisor code which is starting my services is :

[supervisord]
nodaemon=true
[program:adjust_configuration]
command=/usr/local/bin/confd -onetime -backend env
autostart=true
autorestart=true
startsecs=5
[program:copy_setenv]
command=/usr/bin/sh -c "/opt/cmaas/portal/conf/setenv.sh > 
/opt/cmaas/portal/bin/setenv.sh" && /usr/bin/chmod +x 
/opt/cmaas/portal/conf/setenv.sh
autostart=true
autorestart=true
startsecs=5
[program:ssh]
command=/usr/sbin/sshd -D
autostart=true
autorestart=true
startsecs=5
[program:apache]
command=/usr/sbin/httpd -D FOREGROUND
autostart=true
autorestart=true
startsecs=5
stdout_events_enabled=true
stderr_events_enabled=true
[program:tomcat]
command=/opt/cmaas/portal/bin/catalina.sh run
autostart=true
autorestart=true
startsecs=5
stdout_events_enabled=true
stderr_events_enabled=true
[program:adjust_permissions]
command=chown -R cmusr:cmusr /opt/cmaas
autostart=true
autorestart=true
startsecs=5
[eventlistener:logging]
command=supervisor_logging
events=PROCESS_LOG

Tried to create the container manually and put syslog driver but still is showing json-file

docker inspect -f '{{.HostConfig.LogConfig.Type}}' container json-file

any ideas how to get a supervisor to send its logs to a specific syslog server and how to do it?

VIC Version 1.4.3 VCenter and ESX 6.5

yg382 commented 5 years ago

when syslog will be supported for docker create/run ???

renmaosheng commented 5 years ago

Hi have you tried to create VCH using "--syslog-address" to redirect to syslog server? https://vmware.github.io/vic-product/assets/files/html/1.4/vic_vsphere_admin/vch_general_settings.html#syslog

yg382 commented 5 years ago

yes.Note: I can't see any connectivity problems between the vch host (y.y.y.y) and the syslog server (x.x.x.x)

Feb 14 2019 07:30:19.887Z INFO diffID for layer 36ea048a1c37ce224fed626584b2df3a3a34fd7d602a01301364d8822171d0a0: sha256:3444e243271421dbf2df714b76c0ce39daf5c2861f7761e7f03f171d4c60bf2a Feb 14 2019 07:30:19.887Z INFO Path: /tmp/https/10.194.100.93/v2/images/lbext_01/latest/36ea048a1c37ce224fed626584b2df3a3a34fd7d602a01301364d8822171d0a0/36ea048a1c37ce224fed626584b2df3a3a34fd7d602a01301364d8822171d0a0.tar Feb 14 2019 07:30:20.135Z INFO diffID for layer c28519c8a82c3ccf6bca8932f176ee0cf220828723d2b3897e051d3fd0342a95: sha256:ff4dfefed3ed37857c168847638d4a53962aef8c1ab84cdf31efc41205f717c8 time="2019-02-14T07:30:20Z" level=error msg="syslog write failed: write udp y.y.y.y:34168->10.194.103.13:514: write: connection refused" time="2019-02-14T07:30:20Z" level=info msg="trying to connect to syslog server" time="2019-02-14T07:30:20Z" level=info msg="trying to connect to udp://x.x.x.x:514" time="2019-02-14T07:30:20Z" level=info msg="successfully connected to syslog server" Feb 14 2019 07:30:20.351Z INFO diffID for layer b7180bca3575f9aca06f750755858f7f663b615f6374d7043f02499d42328635: sha256:e691203f0b17c6f513da7c08f8d4f214fd370cf4176af3922189b10368e46f79 time="2019-02-14T07:30:20Z" level=error msg="syslog write failed: write udp y.y.y.y:47209->10.194.103.13:514: write: connection refused" time="2019-02-14T07:30:20Z" level=info msg="trying to connect to syslog server" time="2019-02-14T07:30:20Z" level=info msg="trying to connect to udp://x.x.x.x:514" time="2019-02-14T07:30:20Z" level=info msg="successfully connected to syslog server" Feb 14 2019 07:30:20.594Z INFO diffID for layer a718dd7f6bded0af54bc4551537e818214bafea5c23ae3ea368bdbc06b0f249d: sha256:fa8db2c60469f01cb71e86626a123083bbcd473c066624665400286644d5be11 time="2019-02-14T07:30:20Z" level=error msg="syslog write failed: write udp y.y.y.y:47224->10.194.103.13:514: write: connection refused" time="2019-02-14T07:30:20Z" level=info msg="trying to connect to syslog server" time="2019-02-14T07:30:20Z" level=info msg="trying to connect to udp://x.x.x.x:514" time="2019-02-14T07:30:20Z" level=info msg="successfully connected to syslog server" Feb 14 2019 07:30:20.855Z INFO diffID for layer a99b4d1eec652638b65a1e14ea057b444ea2fa8cc3f3601552fb1a0f628c8504: sha256:d22f3dd317bd4c6ee5677a84bfcb4947858313d59e5087268cff6b471d8de479 Feb 14 2019 07:30:21.104Z INFO diffID for layer 8a219342d604203c68d3cc17d99c760d85f2464da5fdd4f0855602c840a59e63: sha256:1bdbe70d79430057c165e119feb38d8a9e437b803e7adb4dce94a0dcfef1eff3 Feb 14 2019 07:30:21.366Z INFO diffID for layer c56e0e1d6c462b4a4c3790732a7180a16652004bbd34ccb169ea7721b9533b47: sha256:8305f9048315b881605723ff34eb15753b5b84edf68dfc386428f472595fb8c9 time="2019-02-14T07:30:21Z" level=error msg="syslog write failed: write udp y.y.y.y:59962->10.194.103.13:514: write: connection refused" time="2019-02-14T07:30:21Z" level=info msg="trying to connect to syslog server" time="2019-02-14T07:30:21Z" level=info msg="trying to connect to udp://x.x.x.x:514" time="2019-02-14T07:30:21Z" level=info msg="successfully connected to syslog server"