Pre-Configured GitLab and Management #28

Closed russleyshaw closed 7 years ago

File: gitlab/base/*

A script/system should be developed to allow us to distribute a base GitLab image. This would be useful because it would allow us to rely on pre-added users/tokens/repos etc.

Most likely there will be some sort of image file that contains the preconfigured image (such as base_gitlab.tar)

Depending on Docker functionality the exact script features can be changed, but the basic idea is

[x] Create new GitLab environment
[x] Save current GitLab environment to file
[x] Load file as current GitLab environment

The script:

[x] Should create a "fresh" GitLab image as the base_docker image
[x] Should load a previously exported image as the current base_docker image
[x] Should run the current base_docker image as a container
[x] Should save the base_docker container as an image
[x] Should export/save base_docker image

Other requirements

Should prompt user Are you sure? [y/N] before applying any changes to exported GitLab file
Update local README.md with specific information about preconfigured GitLab, such as usernames, passwords, tokens, etc

Hints

docker save does not save the file system
docker export does not export any docker metadata (such as CMD, exposed ports, etc)
The admin user is root. The password is requested on first login. Use mstsiggame
To test, you should only need to define a root password, export the container, reload the image and container, and login as root with the password you defined earlier. ~Its not good practice to commit large binary objects like tars but I honestly do not see a way around this considering Docker does NOT give us the easy utilities to define a pre-configured workspace~ These generated files are GB in size. Even if it was considered okay to push these files, it would take individual developers FOREVER

This might be a futile effort considering its incredibly likely we generate image files that are GB in size. Most likely we will need to host them on the siggame@mst.edu Google Drive or on r99 and provide links to them in the README.

https://docs.docker.com/engine/tutorials/dockervolumes/#/backup-restore-or-migrate-data-volumes

I think this may be a way forward.

Try it out. Currently there are 3 volumes used by GitLab to store data. I dont know if that is the entirety of the data that gitlab uses, but I can confirm that I can run gitlab multiple times with the same configuration. Of course the issue here is having it keep that configuration from a clean workspace. Maybe you can export those 3 volumes to 3 respective tar files.

I think all of the gitlab specific information is contained between these three volumes

VOLUME /etc/gitlab
VOLUME /var/log/gitlab
VOLUME /var/opt/gitlab

Try exporting these to .tars, then import them into a fresh GitLab image and see if changes persist.

https://gitlab.com/gitlab-org/gitlab-ce/issues/14740

after googling around maybe this will be the thing we should do? I am still looking into this though.

I think a more docker specific approach might be better, and its basically doing the same thing as cloning the volumes because we know that should work since otherwise, a dockerized gitlab would not be able to survive reboots. This technique could also be applied to other services if we want, for example, a pre-configured database.

But in the end, whatever works would be nice.

https://hub.docker.com/r/gitlab/gitlab-ce/~/dockerfile/ https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/docker/assets/wrapper

My current solution hooks into the post reconfigure 'hook' that this wrapper script has in it. This wrapper script is called every time that docker restarts the container so it is possible to restore backups fairly easily this way.

Maybe at the end of the day this is what we want anyways? This will make testing like 1000000x faster, since gitlab takes ~5 minutes to prop itself up even between warm restarts. I can't tell if it's services or the reconfigure which costs more in the wrapper script.

Another option may be to just redefine what the wrapper script does. This will give more control over what we want done each time the container is restarted.

(I still haven't made that SO post... I want to exhaust all my options first. I'm close to giving up though).

http://docs.projectatomic.io/container-best-practices/#planning_storage

This seems like a good resource with good recommendations. I think reaching out on SO or some docker irc channel is going to be the best bet for a 'good' solution to this problem though.

Crazy idea: use an actual VM (such as VirtualBox/Vagrant). Docker really isnt offering us anything right now in the realm of functionality for GitLab. This would be similar to exporting/importing the application as an .ova file in VirtualBox's import/export menus.

What do you think? Or is the fluidity of docker-compose too important to give up just yet?

https://www.vagrantup.com/docs/cli/package.html https://www.virtualbox.org/manual/ch08.html

Another idea: Use docker export but manually begin the gitlab process

Update

after investigating a bit of this on my own, this seems like the most reliable and simplest solution. We may want to make sure we add a runGitLab script

Mainly completed by #31 PR additional features/bugs should be added individually

siggame / coliselia