search

sonic-net / sonic-buildimage

Scripts which perform an installable binary image build for SONiC
Other
740 stars 1.43k forks source link

Push containers' logs to journald instead of json-file driver #3645

Open 3Hren opened 5 years ago

3Hren commented 5 years ago

Hi!

During playing with SONiC we encountered an issue when all disk's space was consumed by docker containers' logs. I saw the https://github.com/Azure/sonic-buildimage/pull/563 and seems it must prevent such behavior, but seems like it's not. For example swss container still pushes its logs to the bind mount/var/log/swss:/var/log/swss:rw.

There are 2 questions:

  1. Isn't it better to configure default docker log driver to something like journald, which is system-wide and can be externally configured to collect as much logs as switch admins want?
  2. Maybe it will be more general to route swss logs to the log driver (via stdout or other stuff) to apply the common logs-rotation-logic?

I can send a PR if you like.

lguohan commented 5 years ago

@jleveque , what do you think?

jleveque commented 5 years ago

First, I'd like to hear more about the intended behavior from @qiluo-msft who created PR https://github.com/Azure/sonic-buildimage/pull/563.

qiluo-msft commented 5 years ago

@3Hren Could you give some details on "... an issue when all disk's space was consumed by docker containers' logs"? We would like to know which component contributes to the disk consumption.

/var/log/swss is used to store swss and sairedis recording, and they are rotated.

  1. journald is a promising, please help exploration or pull request

  2. What is the benefit of " route swss logs to the log driver" ? @kcudnik may comment.