Initial working version on Windows.

[tokejepsen]

I have tried to minimize the amount of code changes, and mainly just focused on a guide to get setup.

Let me know what you think.

• Related Issue

[tokejepsen]

The idea is to access the working version and then reduce the amount of commands for entry.

[mottosso]

Cool, yeah makes sense.

[mottosso]

We'll need to pick an OS.

Our choices I think are:

1. CentOS 7
2. Ubuntu 17
3. Alpine 3.7

These are the latest of each, which I'm not entirely sure is a good idea. I'm also including "Alpine" which seems to become a growing standard for containers (due to it's small size?). The benefit there (I presume) is that we'll be able to find more help about it, specifically for Docker.

Having said that, CentOS (RedHat based) is more familiar in VFX, which means someone with experience in that could transition their knowledge into the associated Dockerfile.

Now the key questions to ask I think is:

1. Do we expect users to view/edit the Dockerfile?
2. If so, what kind of users are they? VFX artists, or Docker fanatics?

Both of those questions affect our choice.

For one, if they aren't going to look at it (perhaps there simply isn't any need, this is solely a means of distributing Avalon and doesn't/shouldn't affect their environment or technology stack) then we may be better off with Alpine as there is an example Dockerfile we could draw inspiration from. Having said that, it probably doesn't matter that much as odds are every other distribution of software that we use are likely to use something else. Mongo for example uses Debian (akin to Ubuntu).

Ubuntu is perhaps a good choice as it is both somewhat used in VFX and have a large community in server-software outside of VFX, which means we are more likely to find help.

[mottosso]

Looking through the existing Samba and Mongo containers, there a lot of stuff for us to replicate.. some including permissions and authentication (a.k.a. boring but important stuff).

It might be worth looking into whether there is some way we can make a Docker Compose install easy.

The benefit of that is that we can leverage these containers as-is, rather than try and replicate them (in a potentially different OS). The con is that there isn't (that I know of) a command-line option or hosting service for docker-compose files.

$ # Step 1
$ wget https://github.com/getavalon/docker/docker-compose.yml
$ # Step 2
$ docker-compose up

Which assumes wget (or something like it) in addition to Docker, and that extra step.

[tokejepsen]

Sorry, I might be missing a beat here.

Are you concerned about that we are using different OSes in different containers?

[mottosso]

Sorry, I might be missing a beat here.

Which part are you referring to specifically? If we go for a single Docker container, as opposed to Docker Compose, then we'll need to pick an OS in which to install Samba and Mongo. With Compose, we won't have to (and that's ok).

[mottosso]

Here's a one-liner that works right now, it just isn't pretty.

$ docker run --rm -v $(pwd):/work -w /work appropriate/curl -O https://raw.githubusercontent.com/getavalon/docker/master/docker-compose.yml && docker-compose up

Then you'll have your files at \\192.168.99.100\avalon

If you haven't built avalon/docker:0.1 yet then it'll complain about it not existing. Once we've deployed it to Docker Hub, it would get automatically downloaded.

[mottosso]

I've been looking into Docker for Windows (the HyperV version which is also the one they encourage you install from their website) and it looks a little more tricky to share a directory there.. primarily as that Docker binds directly to localhost which is of course already occupied by either Windows itself or the Docker runtime.

[mottosso]

Ok, I've made some progress but also run into a problem.

Firstly, you can now run a single command on any computer with Docker Toolbox and all is well.

$ docker run --rm -v $(pwd):/work -w /work appropriate/curl -O https://raw.githubusercontent.com/getavalon/docker/master/docker-compose.yml && docker rm $(docker ps -aq --filter name=getavalon) 2> /dev/null && docker-compose up -d

There is also a development option, for mounting the git repo rather than the distributed Avalon data container.

$ git clone ...
$ cd docker
$ docker-compose -f docker-compose.yml -f docker-compose-dev.yml up

However this only works on Docker Toolbox, not the "standard" HyperV version. That's all there is to it. It isn't possible to share via Samba on any other port than 445 for Windows.

That is kind of a showstopper. I'd rather not have to ask anyone to install such a specific dependency, as it would also require anyone with an existing Docker install to uninstall it, unless it was Docker Toolbox. For Linux and OSX? The same problem applies. If the user has Samba already on his system, he'll need to disable or uninstall it first. That's no good.

For this to work, we need some way of exposing data in a container to the host. All documentation and help I've found so far has been about sharing it the other way around; from host to container.

There is one last option I was going to try. Which is to do a one-time "download" of files in the container to a mapped drive.

$ docker run -ti --rm -v %cd%/avalon:/avalon getavalon/docker:0.1
Copying files...
Running...

The files would then reside at /_avalon inside of the container, and be copied to /avalon when run. It's not pretty and would pollute a users disk with these files whereas with Samba it would merely mirror the files to an accessible network location.

On the upside however, network sharing the files do have a performance impact. I found that launching Avalon takes a good few seconds. Being copied to the local directory would be much faster.

[mottosso]

Looking deeper, the one-time copy option seems to be the only cross-platform route.

• https://github.com/moby/moby/issues/36974#issuecomment-387053229

There are other options, such as FUSE (on Unix) and mounting a container directly via some Linux magic but nothing comes close to the simplicity of Samba sharing a folder from inside the container to the host..

What a shame! Will keep looking..

[mottosso]

Ok, so it looks like Docker Toolbox is the only option and it maybe for the better.

1. Docker Toolbox has a dedicated IP, the IP of your VM, e.g. 192.168.1.15
2. Docker Toolbox is supported in Windows 10 < Pro from April 2018, such as 10 Home, 7 and Vista.

That would "solve" the problem for Windows, the question remains whether the Samba method applies to Linux..

[mottosso]

Noticed the one-liner above was using getavalon rather than your PR.

Here's what works.

$ docker run --rm -v $(pwd):/work -w /work appropriate/curl -O https://raw.githubusercontent.com/tokejepsen/docker/master/docker-compose.yml && docker-compose up -d

If it complains about existing containers, you can erase them with.

$ docker rm $(docker ps -aq --filter name=getavalon)

Linux

The samba approach does work on Linux.

$ docker run \
    -ti \
    --rm \
    -p 139 \
        -p 445 \
        --name samba \
        -v $(pwd)/avalon:/avalon \
        dperson/samba \
            -s "avalon;/avalon;yes;no;yes;all;none;all" \
            -u "myuser;mypass"

$ sudo mkdir /mnt/avalon
$ sudo mount -t cifs -o user=myuser,password=mypass,vers=2.1 //localhost/avalon /mnt/avalon

The vers=2.1 is necessary due to a recent change to permissions.

The above assumes no existing Samba is running, which can also be worked around by connecting to the Docker instance via its IP.

$ sudo mount -t cifs -o user=myuser,password=mypass,vers=2.1 //172.17.0.2/avalon /mnt/avalon

The latter is what Docker for Windows doesn't (seem to) allow, which is a shame because it's the only way to not require the user to have the Samba ports open.

In this case the remainder I think is getting rid of Compose, such that we get this.

$ docker run -d -p 445:445 -p 27017:27017 --name avalon getavalon/docker:0.1

@tokejepsen thoughts?

[tokejepsen]

Sorry, that is a lot of info to digest :)

I initially thought that having the samba.sh and mongo.sh was part of the plan, hence is why I didn't get why we needed to pick an OS. I guess using multiple containers is just a manual kind of Docker compose, so we might as we'll use compose if using multiple containers.

If we can reduce the setup to one line, or at least just have Docker Toolbox as a single dependency to install, then I don't see why we shouldn't use compose. For the end-user it'll be same, but easier to maintain.

I have very well missed something with your experiments about what works on Windows and Unix, so let me know.

[mottosso]

I guess using multiple containers is just a manual kind of Docker compose, so we might as we'll use compose if using multiple containers.

Yes, exactly. The fork in the road I had in mind was to either use multiple containers, or use a single container with everything built-in. For that single container, we'd need to pick an OS. For the time being, let's stick with Ubuntu.

I guess using multiple containers is just a manual kind of Docker compose, so we might as we'll use compose if using multiple containers.

The reason for preferring a single container, is:

1. The command is a single command, and makes sense e.g. docker run getavalon/docker:1.0 as opposed to the monstrosity above
2. Upgrades are atomic and clear, it's either getavalon/docker:1.0 or getavalon/docker:1.1, as opposed to multiple containers, each with their own version. The docker-compose.yml file is where versions are "grouped", but it's not clear from either the file or when it's running which file it is or when it's from. (it doesn't have a version)
3. Starting and stopping is atomic, it's docker stop avalon as opposed to docker-compose down which can stop some containers not others, depending on whether your docker-compose file is up to date with the containers you've got.
4. There is no local files; the docker-compose.yml must be downloaded stored somewhere, which means it must be taken care of, and ideally re-used, to avoid (3) (and generally make sense out of what's actually running). The downside is that when we could ask the user to run docker run from anywhere and get started using Avalon, with Compose we would need to ask them to move to a certain directory and keep track of the docker-compose file (and potentially explain what Docker Compose is).

With that in mind, my bet is on a single container.

At the very least, now that we've got a setup for Compose, we should make one for a single container so we can pick the one that makes the most sense, once all cards are on the table.

I've made some minor progress on a single container, but there's work left to do. I haven't yet figured out how to simply run samba in our container. The reference container is using special magic that I haven't been able to replicate.

You are welcome to give it a go, I'll be hands-off for a day or two at least.

[tokejepsen]

What is the current problem with samba and the Dockerfile?

Just built it locally here, and I can access samba on "\192.168.99.100\avalon".

[mottosso]

Just built it locally here, and I can access samba

Woot!

[mottosso]

In that case, we only need Mongo and we're good to go. Up for it?

[tokejepsen]

I'll try and continue with Mongo, hopefully later today :)

[tokejepsen]

There is a working version now.

I have just copied the Mongo container and fused the entrypoints together with a shell script. Samba gets run in the background as a daemon.

I don't know what is the best way of moving forward from there though. We could try to strip the Dockerfile down to minimum required commands for easier maintenance or some how organize it so we know how the Samba and Mongo container were fused together?

[tokejepsen]

Sorry, missing a piece. Its incoming now.

[tokejepsen]

Don't know how I could have pushed that last commit, but its all there now.

[mottosso]

I don't know what is the best way of moving forward from there though. We could try to strip the Dockerfile down to minimum required commands for easier maintenance or some how organize it so we know how the Samba and Mongo container were fused together?

Yes, for sure. The original Samba container had an entry file for handling options from the command-line, I stripped those and merged it with the Dockerfile. We should do the same for the Mongo entrypoint. It's huge, and we should really only need to install it, set some environment variables and launch it.

Having a look at this now.

[mottosso]

Ok, it's been stripped, but I'm having issues connecting and am not sure it's just me or whether it's something to do with the image.

Could you run this and see whether everything still works for you?

[tokejepsen]

No, I cant connect to Mongo with the strip down. Samba is working fine though here.

[tokejepsen]

Could maybe be the `docker-entrypoint.shthat needs reinstating?

[mottosso]

Did you expose the port?

$ docker run -ti --rm -p 27017:27017 -p 445:445 -p 139:139 getavalon/docker:0.2

Could maybe be the `docker-entrypoint.shthat needs reinstating?

Preferably we'd have nothing but this one Dockerfile. Dislike calling out to other files. Maybe there's a call in there that we need and could add to our Dockerfile.

[tokejepsen]

Did you expose the port?

Yeah.

Maybe there's a call in there that we need and could add to our Dockerfile.

Yeah, probably some magic in that huge shell script.

[tokejepsen]

Think I have found the magical argument for mongod from docker-entrypoint.sh. Could you confirm this?

Looks like most of the code in docker-entrypoint.sh are for customizing the container at runtime.

[mottosso]

Works here.

One thing that didn't work still was the samba share. After some investigation, I think the problem lies with recent versions of Windows 10, requiring a username and password for samba shares. So I've added this, default to avalon/default to be edited via docker run -e password=mypass.

With that, this now works out of the box! I think it's merge time, and onto the next big thing, which is testing and using.

[mottosso]

As I pushed the button, I realised we need to also update the README. :)

[tokejepsen]

With that, this now works out of the box! I think it's merge time, and onto the next big thing, which is testing and using.

Great!

As I pushed the button, I realised we need to also update the README. :)

Haha, yeah. Always something. I guess we cant do the end-user docs, since we missing the container on Docker Hub?

[mottosso]

Haha, yeah. Always something. I guess we cant do the end-user docs, since we missing the container on Docker Hub?

Good point, setting up automatic upload now.

[mottosso]

Something is up with Docker hub at the moment, delaying.