Notebooks to upload/download marine footage, connect to a citizen science project, train machine learning models and publish marine biological observations.
PR #240 introduces a hack to update the code in the master container. This works and is fast, but it does increase the size of the image every time it is done, since it adds an extra layer. But since we sometimes rebuild the whole image due to which all extra layers get reset, and we do not know how big 1 extra layer is, this maybe does not matter at all. To not use this hack but solve this rebuilding of the master docker image without rebuilding everything, we need some improvements in the Dockerfile, which are listed here below. These improvements are in general good improvements and can also be done while keeping this hack.
(Because this Hack might be quicker than the solution, since we have the container of the ffmpeg that is big and we need to rebuild all the time.)
Some improvements that can be made are:
[ ] Cache the ffmpeg part in image and push this so that we can simply pull in on a rebuild. Right now it has to rebuild this container every time. (See this on how to push 2 images in 1 docker file: https://docs.docker.com/engine/reference/commandline/build/#target)
[ ] Move the COPY from builder to the end, so that we can reuse the layer with all pip installations.
[ ] Move the COPY . ./kso to the end, and in the beginning only copy the requirements, so that the layer with all the pip install does not need to be rebuild on every code change.
[ ] split the run apt-get into 2: 1 with the run apt-get and 1 with the pip installs.
[ ] Look for another option than building the ffmpeg container ourselves, because or rebuilding, or pulling it in both takes more time than if we could just get it from somewhere else.
The environment variables can stay at the end, they are small and quick.
The goal is to get the COPY . ./kso as the last layer, so that when we pull and build an image, everything except for this copy layer can be reused. If we make that possible, then we can remove the Hack. (If it is equally fast as the hack)
PR #240 introduces a hack to update the code in the master container. This works and is fast, but it does increase the size of the image every time it is done, since it adds an extra layer. But since we sometimes rebuild the whole image due to which all extra layers get reset, and we do not know how big 1 extra layer is, this maybe does not matter at all. To not use this hack but solve this rebuilding of the master docker image without rebuilding everything, we need some improvements in the Dockerfile, which are listed here below. These improvements are in general good improvements and can also be done while keeping this hack. (Because this Hack might be quicker than the solution, since we have the container of the ffmpeg that is big and we need to rebuild all the time.)
Some improvements that can be made are:
The environment variables can stay at the end, they are small and quick.
The goal is to get the COPY . ./kso as the last layer, so that when we pull and build an image, everything except for this copy layer can be reused. If we make that possible, then we can remove the Hack. (If it is equally fast as the hack)