Add metadata cell to notebooks

MaxTuecke commented 5 months ago

Closes #362

Overview

To make notebooks location independent, save the notebooks base-image and requirements at the top cell of every notebook, rather than in local data

Notebooks will now always have the top cell as the garden metadata cell.
Requirements files will now be read and their contents saved in the metadata rather than just saving the path.
All notebook templates were updated with the the correct base image in the garden metadata cell
notebook start will automatically add the garden metadata cell if its missing.

Discussion

Adds new helper functions _add_notebook_metadata_cell, _set_notebook_metadata and _get_notebook_metadata, in notebook.py to read / edit the garden metadata cell.

Adds new helper functions _read_requirements_data and _save_requirements_data in notebook.py for reading and writing new requirements data format.

Removed _put_notebook_base_image and _get_notebook_base_image from local_data.py

The notebook publish functionality was slightly changed. Previously, publishing a notebook with a new base-image or requirements updated the user's local notebook in addition to publishing. This is no longer the case. Now when a user publishes a notebook using a different base image or requirements file, the notebook that gets uploaded will have the new metadata saved in it, but publish will not modify the user's local notebook. This choice was made because a user should not be able to accidentally change their local notebook if they are trying to publish a variation of it using publish's requirements or base-image arguments. If a user wishes to make changes locally, 'notebook start' should be used.

Testing

Manually tested notebook start, notebook debug and notebook publish commands and ran remote execution of a newly published garden and endpoint.

Documentation

No docs changes.

📚 Documentation preview 📚: https://garden-ai--404.org.readthedocs.build/en/404/

codecov-commenter commented 5 months ago

Codecov Report

Attention: Patch coverage is 56.61765% with 118 lines in your changes are missing coverage. Please review.

Project coverage is 76.63%. Comparing base (037216d) to head (7428a14). Report is 8 commits behind head on main.

Files	Patch %	Lines
garden_ai/app/notebook.py	4.76%	100 Missing :warning:
garden_ai/notebook_metadata.py	83.48%	18 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #404 +/- ## ========================================== + Coverage 75.25% 76.63% +1.38% ========================================== Files 45 48 +3 Lines 2683 3017 +334 ========================================== + Hits 2019 2312 +293 - Misses 664 705 +41 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

MaxTuecke commented 5 months ago

Codecov Report

Attention: 174 lines in your changes are missing coverage. Please review.

Comparison is base (037216d) 75.25% compared to head (2a7db4f) 72.04%.

Files Patch % Lines garden_ai/app/notebook.py 5.43% 174 Missing ⚠️ ❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files ☔ View full report in Codecov by Sentry. 📢 Have feedback on the report? Share it here.

Need to add unit tests for notebook helper functions still.

Garden-AI / garden