search

valohai / valohai-utils

Python helper library for Valohai
MIT License
2 stars 2 forks source link

Support notebooks in YAML generator (fixes #61) #62

Closed JuhaKiili closed 3 years ago

JuhaKiili commented 3 years ago

This PR adds notebook source file support for the YAML generator that the Valohai CLI is calling.

You can create YAML step from Python file today:

vh yaml step hello.py

Now with valohai-utils support for notebooks, you can do the same with notebooks:

vh yaml step hello.ipynb

This will generate the YAML step that runs a notebook using Papermill. The notebook is assumed to use valohai-utils to define parameters and inputs.

Note: This PR has some copy-pasta code from Jupyhai, related to parsing source code from notebooks. We don't want valohai-utils to be dependent on Jupyhai, but Jupyhai is already dependent on valohai-utils. Once this PR is merged in, we should DRY at the Jupyhai side if possible.