search

google / weather-tools

Tools to make weather data accessible and useful.
https://weather-tools.readthedocs.io/
Apache License 2.0
214 stars 40 forks source link

Find a way to exclude test data when building docker image. #350

Open alxmrs opened 1 year ago

alxmrs commented 1 year ago

I just noticed something. When trying to build a docker image with a clean-ish branch (no non-checked in files), I got this log trace from Google Cloud Build: 

Creating temporary tarball archive of 159 file(s) totalling 190.3 MiB before compression.

I did an experiment: I deleted all the test data folders and re-rand the command to build our docker image:

Creating temporary tarball archive of 124 file(s) totalling 958.1 KiB before compression.

This is quite a huge improvement! In general, I wonder if there's a way where we can not have checked-in test data.

alxmrs commented 1 year ago

Since we git clone weather tools in the docker image, we should also find a way to prevent test data being checked in.

https://github.com/google/weather-tools/blob/main/Dockerfile#L29

I think this could make our image more lightweight!

alxmrs commented 1 year ago

Here's an idea! Let's replace hosting our own test data by using examples from the Xarray tutorial: https://docs.xarray.dev/en/stable/generated/xarray.tutorial.open_dataset.html