search

fluent / fluentd-docker-image

Docker image for Fluentd
https://hub.docker.com/r/fluent/fluentd/
Apache License 2.0
464 stars 351 forks source link

Confusion Over Docker Hub Fluentd vs Official #189

Open james-crowley opened 5 years ago

james-crowley commented 5 years ago

I am trying to work with Fluentd on s390x, but I am a little confused on which Docker Hub I should be pulling from.

On the top of this repo, you have a link pointing to https://hub.docker.com/r/fluent/fluentd. Which seems great, since you have the needed version tags that I want, but when I go to utilize this image I run into a standard arch issue.

standard_init_linux.go:190: exec user process caused "exec format error"

After doing some digging around I see you guys started the progress on getting Fluentd as one of the official images on Docker Hub. https://github.com/docker-library/official-images/pull/3724

Seems like this push was due support of ppc64le with help from @vaibhavsood. But looking at the official Docker Hub page, https://hub.docker.com/_/fluentd, it seems to only support a couple versions of Fluentd and lacks continuous updates like your other Docker Hub page.

I would really love to use Fluentd on s390x, but want access to new updated images and ideally older releases like v1.2.6.

@repeatedly Any input on this issue?

Additionally, there seems to be active talks and development around ppc64le, https://github.com/fluent/fluentd-docker-image/issues/120, thanks to @ghatwala. I would love talk about the possibilities of doing something similar to this but for s390x.

ghatwala commented 5 years ago

@james-crowley

After doing some digging around I see you guys started the progress on getting Fluentd as one of the official images on Docker Hub. docker-library/official-images#3724

The PR responsible to add fluentd docker image as official image in dockerhub is this one - https://github.com/docker-library/official-images/pull/5303 . This image - https://hub.docker.com/_/fluentd/ is a multiarch image supported for s390x, ppc64le as given here - https://github.com/docker-library/official-images/blob/master/library/fluentd#L13 . However doesn't support older versions v1.2.6.

Additionally, there seems to be active talks and development around ppc64le, #120, thanks to @ghatwala. I would love talk about the possibilities of doing something similar to this but for s390x.

For travis support to be added for s390x , there should be similar queue created at travis end similar to the one created for ppc64le .

I am trying to work with Fluentd on s390x, but I am a little confused on which Docker Hub I should be pulling from.

As fluentd is officially multiarch and supported on s390x , check the official jenkins jobs ( for dockerhub) - https://doi-janky.infosiftr.net/job/multiarch/job/s390x/job/fluentd/ running builds on s390x.

repeatedly commented 5 years ago

Docker official image refers this repository so the image contents are same. The merit of official image supports multi architecture. ghatwala mentions this point above.

We want to avoid the regression for official images so we don't update fluentd official image for every release. And for official images, we don't have plan to add older version because newer version is better than older.

james-crowley commented 5 years ago

@ghatwala Thanks for the information about the two different Docker Hub repos.

@repeatedly I know you said you don't plan on adding any older versions, but there are still some projects utilize those versions. Thus I would argue it would be good for the community to add at least some of the "major" older versions being used by projects.

I am trying to utilize the kube fluentd operator(@jvassev), https://github.com/vmware/kube-fluentd-operator, for a client project that I am working on. As of right now they only support Fluentd 1.2.6.

I got Fluentd 1.2.6 built for s390x with pulling down the official Dockerfile from this repo. All it would need is to be pushed to Docker's Official Hub for all this to work flawlessly on s390x.

james-crowley commented 5 years ago

@repeatedly Any thoughts on my post above. Again, I think adding some of the "major" older version to docker hub would benefit the community.

I hope community members who utilize Fluentd start to update their versions, but progress is slow. Thus backporting some of key versions that community projects uses is a good idea.

Please let me know if you are onboard with this issue. If not, please feel free to close the issue. Thank you for all your input so far.

james-crowley commented 5 years ago

@repeatedly Don't mean to spam you, but thought you be interested in this development.

Dumb-init, just add support for s390x and back ported their releases for me, https://github.com/Yelp/dumb-init/issues/188. Thus making the Dockerfile listed in the v1.2 folder, fully compatible with s390x. No modifications are needed to be made to add support for s390x, all that is needed is to have the images be pushed to the official Docker hub.

repeatedly commented 5 years ago

I am trying to utilize the kube fluentd operator(@jvassev), https://github.com/vmware/kube-fluentd-operator, for a client project that I am working on. As of right now they only support Fluentd 1.2.6. Thus backporting some of key versions that community projects uses is a good idea.

Hmm... I see. The problem is fluentd v1.2.6 has thread handling issue of output plugin and it causes weird error for buffer/C-ext library. So we don't recommend to use v1.2.6 for production. fluentd v1 keeps the backward compatibility excluding internal code layout so I assume updating the version from v1.2.6 to latest is easy.

fluentd-docker-image itself sometimes changes Dockerfile for security reasons or Docker update so may need changes for it.

james-crowley commented 5 years ago

@repeatedly Looks like the VMware Operator for Fluentd just updated to Fluentd 1.5.2. I know you mentioned not wanting to push a multi-arch image for 1.2.6 due issues it had, but would you push 1.5.2? Specifically v1.5.2-debian-1.0?

repeatedly commented 5 years ago

Why VMware Operator uses 1.5.2 instead of latest available version in docker official?

https://hub.docker.com/_/fluentd?tab=tags

Docker official provides 1.6.2 and 1.6.2 is more better than 1.5.2. I'm not sure why VMware Operator uses 1.5.2 like half version...

repeatedly commented 5 years ago

One drawback of releasing old version for Docker official is latest tag rollbacks to old version, e.g. 1.6.2 to 1.5.2. This is very confusing for other users.

james-crowley commented 5 years ago

@repeatedly I am not sure why they made the jump to 1.5.2. It might be because of plugin compatibility, I am not the creator so I can't give you an answer for sure.

Is there a reason why you do not want to push other versions to the official Dockerhub? You support 1.4.2 and now 1.6.2, but no 1.5.2?

Also it seems to your latest tag is pointing to one of your older 1.6 builds and not the latest 1.6.2 build. So I am little confused by your statement,

One drawback of releasing old version for Docker official is latest tag rollbacks to old version, e.g. 1.6.2 to 1.5.2.

I think only supporting some versions on the Docker official while having all your versions on Fluentd is confusing for users, hence this issue.

This issue has been opened a month and no progress has been made. If you are unwilling and/or unable to push some of the versions I mentioned, feel free to close this issue.

SehiiRohoza commented 1 year ago

Time flies, but the issue is still in place since 2019:

Please solve it in 2023.

kenhys commented 2 months ago

NOTE:

Docker official image https://hub.docker.com/_/fluentd

SPONSORED OSS https://hub.docker.com/r/fluent/fluentd