eurec4a / how_to_eurec4a

Code examples to get you started with EUREC⁴A data.
https://howto.eurec4a.eu
MIT License
6 stars 20 forks source link

Execution cache #76

Closed d70-t closed 2 years ago

d70-t commented 2 years ago

This PR enables a jupyter-cache based caching of executed notebooks.

The cache will only re-execute notebooks if parts of their code-cells change. Due to the use of IPFS CIDs when retrieving catalog entries, I believe that this change is reasonable: whenever the referenced data may change, we have to update the CID, which will trigger re-execution of the notebook. And the other way around: whenever we don't change the CID, the referenced data can't change, thus the output of the notebook shouldn't change as well.

Note that the cache will be invalidated whenever changes to the requirements are made, thus that we should also be safe when dependencies are update.