In https://github.com/mapbox/robosat/pull/151 we pre-fetched the model's weights into the docker image which makes iterating on training faster. In contrast this changeset is about improving on iteration speed for code development.
Removes unused dependencies to make the image smaller and the docker build process faster; pyosmium switched from Boost.Python to pybind11 and is bundling zlib and expat in their wheel by now. The rtree package only required the C libspatialindex shared object.
Creates a separate layer for the pytorch wheel. Changes a couple times a year and is quite big, should get cached.
Creates a separate layer only for our dependencies. Again they should change only every now and then, installing the python depdency packages is slow, and we should cache this.
Creates a separate layer for the pre-trained weights. By now we do not have the robosat source code in the docker layers yet so we use the underlying raw resnet model.
Finally creates a layer with the robosat code which changes frequently when iterating on code and then building images to push onto my gpu rig.
Because we are now caching layers and carefully ordered them by how often they will change iterating on code is down from tens of minutes to a few seconds copying the new source code into the docker image.
:racehorse: :rocket:
cc @jacquestardie please merge; this works. (please merge with the rebase merge option to get a fast-forward merge keeping a linear git history)
In https://github.com/mapbox/robosat/pull/151 we pre-fetched the model's weights into the docker image which makes iterating on training faster. In contrast this changeset is about improving on iteration speed for code development.
Because we are now caching layers and carefully ordered them by how often they will change iterating on code is down from tens of minutes to a few seconds copying the new source code into the docker image.
:racehorse: :rocket:
cc @jacquestardie please merge; this works. (please merge with the rebase merge option to get a fast-forward merge keeping a linear git history)