Create official Docker image on Docker Hub

atombender commented 5 years ago

As with FoundationDB, there's no official Docker image, despite the presence of a Dockerfile. It'd be nice to have an official image.

apkar commented 5 years ago

I assume you are asking about Docker image for the build. That is what Dockerfile is for. Publishing image for the build can be done quickly. As you can see in the FoundationDB case, Docker image for running is a bit complicated. There is an ongoing PR for this on FoundationDB. Once that is merged we can follow the example.

cc: @AlvinMooreSr

atombender commented 5 years ago

Yes, I'm talking about an official image, not a Dockerfile. The foundationdb repository on Docker.com is where I'd expect to find it.

apkar commented 5 years ago

Ah, sorry I was not clear. We can have two docker images - one for build and one for deployment. We definitely want to do both. Docker image for build will happen soon and image to deploy Document Layer will happen after.

apkar commented 5 years ago

PR on FoundationDB core repo for docker image https://github.com/apple/foundationdb/pull/887

apkar commented 5 years ago

We can follow the example of FoundationDB docker image. There are a couple of details which are not trivial, writing them down here hoping to see better ideas.

FoundationDB client libraries
- Get client libraries on startup - If there is a way to find out FDB server version from the client, then we could get the required client library on startup. This will increase the startup time. As far as I know, it is not possible as there is no easy way to get FDB server version.
- Include client libraries in the image - client libraries will be downloaded during the docker image build process. This would mean we have to update the Document Layer docker image with new FoundationDB releases. I am thinking of going with this approach.
Cluster file
- We could make this an environment variable which could be set at runtime. We have to pass the complete cluster file text as an environment variable. Can't think of any better approach so far.

cc: @brownleej

brownleej commented 5 years ago

I think that the ideal way to get the cluster file and the client libraries will vary based on how someone is deploying the image, e.g. through Swarm, Kubernetes, or some other system. In the main FDB docker image, we have a script for generating a cluster file based on looking up coordinator names in DNS, but that probably won't work outside a simple local testing environment.

platinummonkey commented 5 years ago

In the main FDB docker image, we have a script for generating a cluster file based on looking up coordinator names in DNS, but that probably won't work outside a simple local testing environment.

I think this would be great if it was just a static file. Then the user can wrap whatever they want on how to get that in there: volume mount, some ConfigMap mount, or some script to generate this on the fly.

FoundationDB client libraries

I think this will also be language and build-system dependent. maybe just provide a template for a simple stripped down image tagged by version that people could base off of, or use a template to add to their existing images. Honestly I'd worry about this last $0.02.

Though personally I'd vote for starting simple.

apkar commented 5 years ago

Dockerfile is available now.

@AlvinMooreSr can you push the image to Docker Hub, please?

apkar commented 5 years ago

Thanks @AlvinMooreSr!

foundationdb/fdb-document-layer:1.6.0 is available now.

bencherian commented 5 years ago

@apkar @AlvinMooreSr Is there a plan to push the 1.6.1 release to DockerHub as well?

apkar commented 5 years ago

@bencherian Yeah, we are still in the process of releasing. It's going to happen this week.

bencherian commented 5 years ago

The existing Docker image for the document layer with the "latest" tag on DockerHub is not the same as the latest release 1.6.2. (It's significantly larger and is a few days newer)? What version is this? (There are also tags for 0.0.4 and 0.0.5, despite those not having releases on GitHub...)

apkar commented 5 years ago

Good catch! Thanks for reporting. Seems like we have pushed build docker images to runtime path. We will fix it.

cc @AlvinMooreSr

AlvinMooreSr commented 5 years ago

The docker build images have been removed and the latest tag has been updated with the correct version. http://hub.docker.com/r/foundationdb/fdb-document-layer/tags

wenfang6 commented 2 years ago

l pull fdb-document-layer image from https://hub.docker.com/r/foundationdb/fdb-document-layer/tags, l add fdb.cluster in the image and l start document layer server, but there are some error: Type="StartupFailure" ID="0000000000000000" phase="ConnectToCluster" timeout="120".... l don't know if l missed some key steps

FoundationDB / fdb-document-layer

Create official Docker image on Docker Hub #13