create "official" docker image

[rgerhards]

We should create a great docker image which

• is easily usable in any environment
• showcases what and how rsyslog can do things

We may need to provide more than one image if we identify conflicting needs (e.g. low- vs high-end needs). Among others, features should be

• log data to an (external) volume e.g. container itself is stateless
• provide support for common public endpoints
  • papertrail
  • loggly
  • sematext (ElasticSearch)
• accept messages via
  • tcp (plain, for starters, later TLS -- need to find solution for certs!)
  • udp
  • RELP
• ...

It is assumed that to make everything effortlessly work we will need some feature extensions to rsyslog.

Note: I can and will add feature requests either to this issue directly or link to feature request issues.

[rgerhards]

milestone assigned tenatively

[rgerhards]

@davidelang @radu-gheorghe @hakman @halfer interested?

[taotetek]

@rgerhards what's the goal as far as the rsyslog configuration goes? we're running rsyslog in docker now, but our configuration is very tailored to our environment ( kafka, zeromq, elasticsearch, etc ) - is the goal to include commonly used inputs and outputs in the official container and then allow users to use the official container as the starting point for building a container with their config?

Right now, we bake our custom config into the container as part of our ci/cd build before deploy.

[hakman]

Great proposal. Someone from our team already did something in this direction. https://github.com/megastef/rsyslog-logsene

Let me know what you would need from us..

[rgerhards]

@taotetek It's important that right now I am gathering ideas.

One goal I have is to provide a container that provides almost-ready-to-go logging. So someone could pull the container, do some minimalistic changes and actually use the container in production. A concrete example is what @hakman linked to. In addition, I'd like to auto-detect what is needed. E.g. if a log file volume is actually mounted, then automatically populate it. Silently ignore the file logging part if it is not present. Obviously, I do not even know what all would be possible with it.

A related goal (I think) is to provide a container that you can easily "change a bit" to get to your custom version. I wonder, though, if that is the main target as I would assume that the hardcore folks have no problem setting up the container - it looks indeed extremely easy. But the canned config, finally a good one, is possibly useful in these cases as well.

As (maybe more as) a side-goal I would like to build this in order to experience the problems that come along with containers and get a first-hand idea of the user experience. I think a few tweaks here and there can definitely improve the user experience.

[rgerhards]

@hakman Thanks, https://github.com/megastef/rsyslog-logsene looks interesting. It seems to have all needed for LogSense integration :-)

[halfer]

I wonder if it's worth doing some experimental stuff first, to see how log handling works inside Docker prior to embarking on an "Official Image" sub-project. My experience was that even simple stuff did not seem to work, and I am using Rsyslog in (afaict) a very standard way.

(Not pouring cold water on the idea, just wonder if ensuring everything works in Docker first).

[rgerhards]

I wonder if it's worth doing some experimental stuff first, to see how log handling works inside Docker prior to embarking on an "Official Image" sub-project.

For me, that's part of the process. I fully agree if we provide a "stable official image" it should first and foremost be stable ;-) I'll probably still push some experimental versions for others to see and check, but the descriptions will clearly have EXPERIMENTAL and "do not use in production". From my initial testing I already see a couple of things that should be changed for a really good solution.

I am also undecided if we need a separate git repo for this effort or can do it inside the rsyslog source project. Right now I'll do the initial development in my playground at https://github.com/rgerhards/rsyslog-docker. Sooner or later we need to move it to a somewhat better place.

[rgerhards]

I transferred my personal playground to the rsyslog project. First usable fruit appears: containers we can use in CI. Also hope to get some more attention this way. Now lives at https://github.com/rsyslog/rsyslog-docker

[davidelang]

very interested. I have not used docker before, so I've got some learning to do here.

We need to see how 'fat' the container would be if we try to include support for everything.

[davidelang]

On Mon, 8 Jan 2018, taotetek wrote:

@rgerhards what's the goal as far as the rsyslog configuration goes?

I see two different things.

First, a ready-to-run container, just add config and go. This would replace whatever random thing people find on the Internet, and would get an updated version every time there is a new release

Second, for people who want something more tailored, a container generation script that they can use as a starting point and tweak slightly (ideally with a config menu tool to set at least the most common types of variations) to tailor for their exact needs.

[rgerhards]

working on a logsene container as prototype for what we need: https://github.com/rsyslog/rsyslog-docker/issues/4

[rgerhards]

It looks like we already have what we need, probably just be more vocal about it

[endersonmaia]

I transferred my personal playground to the rsyslog project. First usable fruit appears: containers we can use in CI. Also hope to get some more attention this way. Now lives at https://github.com/rsyslog/rsyslog-docker

why not transfer this issue to rsyslog/rsyslog-docker ?