Resolve environments

[jessjaco]

Here's the current workflow (putting here rather than readme as it may change):

1. Calculate tides using src/calculate_tides.py
2. Filter by tides, create annual mosaics, and calculate water indices using src/water_indices.py
3. Threshold, clean, and vectorize using src/clean_rasters.py

Currently 1 is done locally using he dea-australia/coastlines based docker image. It can be run remotely, as long as the tidal model data is available. It hasn't yet been run in the notebook or kbatch environment.

2 is done using kbatch and the planetary computer docker image (as noted in the yaml) with the addition of dep_tools (which is installed in src/run_on_pc.sh).

3 is done using kbatch and the dea-australia/coastlines based docker image.

We will need to decide if we want to maintain these different dev environments or try to use a single one. Here are confounding issues:

• The tidal models are big, so they make large docker images. In addition, output can not be written to blob storage directly, so kbatch is a no-go. (This could be solved by some manual-uploading perhaps). Nevertheless, we could still standardize on the docker image with the requirement that the tidal models are mounted in a directory.

• I have not been able to install dea_tools in the MSPC environment / atop the docker image, due to dependencies on GCC. However, @sopac has told me:

  I was able to install the dev environment in MSPC via the the following steps.

  1. Generate a requirements file via code analysis usin pipreqs on dep-coastlines code, and clean it up (attached)
  2. conda install python-geohash
  3. pip install -r requirements.txt

• It'd be nice to use the MSPC instance as much as possible, with as little additional installations as possible, so code may function in the notebook environment if possible.

[jessjaco]

Think I resolved this.

[jessjaco]

Solution was to just install gcc on the planetary computer image before installing dea_tools, so hdstats wouldn't error out (Sachin's solution above worked, but using the "official" dea_tools, which doesn't have hdstats).

My only complaint is this makes the docker image rather large. I'm considering maintaining the use of the vanilla planetary computer image when running through kbatch for step 2 above.