sqlboiler docker image?

[dahu33]

Would be nice to publish "official" docker images on hub.docker.com. Happy to help if needed.

[aarondl]

Before having official docker images I think it's important that we re-enable CI. With the coming of Github Actions this will be possible as the blocker is this org will not allow third party services like CircleCI anymore due to the extremely poor granularity on Github OAuth permissions. The CI should create the Docker images after having tested the tag. I think automatic releases without automatic tests is sort of a bad deal.

It's also a bit odd to me to need a Docker image for sqlboiler over say released binaries given that there's already been work done to do asset embedding so you don't need to have anything other than its binaries. Docker only does 2 things for you: Bundles applications (great for ruby/python like interpreted apps where source code is required) and isolates the file system/network. sqlboiler requires you to circumvent the filesystem isolation otherwise the config is not visible and the generated code is emitted into limbo and the packaging isn't necessary because of the aforementioned asset embedding, so what benefit does Docker provide? Maybe it could be argued that it contains all the drivers so it's slightly easier to use? But we could create a zip bundle in the same vein.

What are your thoughts?

[dahu33]

Thanks for looking into this.

Before having official docker images I think it's important that we re-enable CI. With the coming of Github Actions this will be possible as the blocker is this org will not allow third party services like CircleCI anymore due to the extremely poor granularity on Github OAuth permissions. The CI should create the Docker images after having tested the tag. I think automatic releases without automatic tests is sort of a bad deal.

While I completely agree that CI need to be re-enabled asap, I also think this issue is independent of CI for the following reasons: 1) it's not the tag that need to be tested but the branch/commit. If tests are run after the tagging is done, people might start to use the new version before the tests are run. Tagging/releasing should only be done after all the tests have passed. Therefore, IMO, automatically building binaries/docker images on tags is safe (I'm not proposing automatic releases, but automatic docker build of releases). 2) the extremely poor granularity on Github OAuth permissions can be solved by creating "system users", basically a github user that will only have access to a single or set repository in the organization and with particular permissions.

It's also a bit odd to me to need a Docker image for sqlboiler over say released binaries given that there's already been work done to do asset embedding so you don't need to have anything other than its binaries. Docker only does 2 things for you: Bundles applications (great for ruby/python like interpreted apps where source code is required) and isolates the file system/network. sqlboiler requires you to circumvent the filesystem isolation otherwise the config is not visible and the generated code is emitted into limbo and the packaging isn't necessary because of the aforementioned asset embedding, so what benefit does Docker provide? Maybe it could be argued that it contains all the drivers so it's slightly easier to use? But we could create a zip bundle in the same vein.

With docker, a single command can:

• pull and cache locally a particular version of sqlboiler (or just always the latest)
• run sqlboiler independently of the system (CI, developer laptop, etc..) and independently of the OS (linux, mac, windows...)

With docker, we can easily make sure the same version of sqlboiler runs everywhere without having to ask everyone in our team to install or upgrade anything. It's just very convenient and remove a lot of complexity in our CI, in our dev tools and for our on-boarding process.

Hope you will consider reviewing #623 :) I would also be happy to help on re-enabling CI if needed.

[aarondl]

Well, to play some devil's advocate:

• You can always pull a particular version of a binary, the released artifacts (when they exist) will be named: sqlboiler-v3.0.0-linux_amd64 or some such thing, and so it will be trivial to have a script that refers to that binary specifically rather than renaming it to sqlboiler.
• sqlboiler can already be run independently of any system, cross compilation is one of Go's strong suits and we did work to ensure no external dependencies.

Docker is the extreme side of unnecessary when your task is: "run a binary at a specific version for a given OS". The proposed solution looks like:

1. Download and install a daemon whose correct version is only available in repos you must first manually add with apt-add-repository, or install via MSI or Mac App if on those platforms. Also as a side effect install odd networking plugins/virtual filesystems you don't need.
2. Run a script that pins the sqlboiler version for your team inside your repo which will download an operating system (I love Alpine for when I do use Docker, clocking in at 36meg it's pretty reasonable, just not for this) which is 3x the size of the binary you're attempting to run. Docker will then load the virtual file system, mount the directories required (config file and output directories) into the virtual file system, and then run the process in a cgroup which is pretty unnecessary since sqlboiler's not attempting to do anything malicious and doesn't require root in any way.

Stepping back the problem we're trying to solve is fetch & run a binary at a specific version for a given OS. This is simple to do with a shell script which you need regardless because otherwise you cannot pin the version of sqlboiler which is required since Docker's latest tag is completely broken: https://vsupalov.com/docker-latest-tag/

#!/bin/sh

version="v3.0.0"
binspec="${version}-$(go env GOOS)_$(go env GOARCH)"

sqlboiler="sqlboiler-${binspec}"
driver="sqlboiler-psql-${binspec}"
# Can delete this if you don't use windows anywhere
if [ $(go env GOOS) = "windows" ]; then
  sqlboiler="${sqlboiler}.exe"
  driver="${driver}.exe"
fi

if [ ! -e "${install}" ]; then
  url="https://github.com/volatiletech/sqlboiler/releases/download"
  wget "${url}/${version}/${sqlboiler}" -O "${GOBIN}/${install}"
  wget "${url}/${version}/${driver}" -O "${GOBIN}/${install}"
fi

${sqlboiler} --wipe ${driver}

This of course is much more complicated than the following script, but this is only the case if you already have Docker installed.

#!/bin/sh

version="v3.0.0"
docker run --rm -it \
    -v "paths:paths" \
    -v "sqlboiler.toml:path/sqlboiler.toml" \
    "sqlboiler:${version}" sqlboiler --wipe psql

Note: Both my script and docker leave around plenty of outdated versions of things, but Docker's can only be bigger or equal in size.

A third solution which is probably the best of both worlds is a tool like this: https://github.com/twitchtv/retool which solves this exact problem but without a gigantic dependency like Docker. What do you think about an approach like this instead?

[aarondl]

@dahu33 Apologies. I think I'm just getting in the way here for not a very good reason (personal opinion differences). Perhaps we could split some of that difference here. Let me at least be gracious enough to review your changes which I'll do now. It's likely that this will land, but it's unclear whether or not there will be official docker images published on docker hub.

[dahu33]

@aarondl I do mostly agree with you but I still think there is some use-cases for a docker sqlboiler image :). Providing one doesn't prevent any of what you said, it just give the users more options.

In any-case, thank you very much for reviewing my PR.

[jwilner]

I'll just chime in that docker is of course a massive dependency, but it's also an extremely common dependency in the modern software toolbox.

Our generation logic for SQLBoiler already relies on docker for spinning up an ephemeral DB to which to apply our migrations and generate the code from. I suspect that's a pretty common pattern with SQLBoiler.

Providing a canonical docker container is a common way to solve this sort of problem, and my team for one would immediately use it. There's no one saying you have to use Docker. Won't this just ease adoption of this great tool for teams that choose to use docker like this, and won't others who choose not to be unaffected?

[aarondl]

@jwilner None of what you're saying is untrue and it's a good summary of the state of development in general surrounding Docker.

There is an underlying assumption however that we're trying to ease adoption or even achieve more adoption and this is not correct. If there was a goal for adoption, it would be to reduce how many people use the project.

Given that there's no actual reason to add a Docker container other than for the ease of use of our current users, and the current users have spoken and I'm trying to simply stay out of the way on this front.