qiime2 / provenance-lib

QIIME 2 Provenance Replay Tools
BSD 3-Clause "New" or "Revised" License
3 stars 4 forks source link

Adds Jupyter Notebook replay driver #105

Closed ChrisKeefe closed 11 months ago

ChrisKeefe commented 1 year ago

Like it says on the tin, this enables provenance replay to produce Jupyter Notebook (.ipynb) files.

Notes

Draft status

This code has been extensively playtested (and the CLI and Python drivers have been unit tested and diffed against regressions), but no unit tests have been written for the new driver at this time.

Review comments and playtesting feedback are welcome.

ChrisKeefe commented 1 year ago

I incorporated the notebook driver into some basic tests this evening, and added a set of tests covering line breaks in comments which is a kinda gory place in the code to deal with. This changeset still introduces a small loss in coverage, as the notebook driver isn't tested against e.g. parsing multiple results, or certain metadata handling paths.

I need to wind down work on this PR for now - feel free to add those tests, playtest and merge, or make any changes required to get this across the finish line. Thanks, CK

ChrisKeefe commented 1 year ago

Looks like there's a build issue related to a missing seed-environment-conda.yml - lmk if there's anything needed on my end to resolve.

Traceback (most recent call last):
 == Using inputs == 
{
  "conda-activate": "source '/home/runner/work/provenance-lib/provenance-lib/alp-base-env/etc/activate.sh'",
  "seed-environment": "./distro/2023.9/core/staged/seed-environment-conda.yml",
  "conda-build-config": "conda_build_config.yml",
  "channels": "[\"./built-channel\"]",
  "skip-first-channel": "false"
}
 == Evaluated keyword arguments == 
{
  "conda_activate": "source '/home/runner/work/provenance-lib/provenance-lib/alp-base-env/etc/activate.sh'",
  "seed_environment": "./distro/2023.9/core/staged/seed-environment-conda.yml",
  "conda_build_config": "conda_build_config.yml",
  "channels": [
    "./built-channel"
  ],
  "skip_first_channel": "false"
}
 == Executing == 
  File "/home/runner/work/_actions/qiime2/action-library-packaging/beta/make-conda-config/bin/make-cbc.py", line 34, in <module>
    ActionAdapter(main)
  File "/home/runner/work/_actions/qiime2/action-library-packaging/beta/src/alp/common.py", line 39, in ActionAdapter
    results = function(**kwargs, **extras)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/qiime2/action-library-packaging/beta/make-conda-config/bin/make-cbc.py", line 11, in main
    with open(seed_environment) as fh:
         ^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: './distro/2023.9/core/staged/seed-environment-conda.yml'
Error: Process completed with exit code 1.
gregcaporaso commented 11 months ago

@ChrisKeefe, we're going to close this PR, but it's referenced from the framework where we'll ultimately integrate this. We'll need to work on migrating this to a framework PR on our end.