Open bt-maps opened 3 years ago
You can use inputs to parameterized Livebook and fill those in the UI. Because our inputs use the IO API, they also work when the notebooks run in the CLI (i.e. we will request the user to fill info in as it happens). So this functionality is already in place but in a way.
The biggest issue is that we don't have an option to execute notebooks via the CLI.
Depends on https://github.com/livebook-dev/kino/issues/54.
I was hoping to bump this issue as I'm currently working on a pretty novel idea using Livebooks in an educational manner and would like a way to execute notebooks via the CLI in order to automate grading code output. Any help would be appreciated! 😄
Hey @houllette, this is not a priority for us right now, but I put together a script for basic evaluation here. It needs to be run within the Livebook mix project:
git clone https://github.com/livebook-dev/livebook.git
cd livebook
mix dev.setup
# Download the script and use as:
mix run livebook_run.exs ./path/to/notebook.livemd
Hopefully this helps or gives you a starting point :)
Hey @jonatanklosko - Sorry to bother you again after so much time as elapsed since you wrote the aforementioned script. I was just starting to work with it more and am running into issues - I tried debugging it myself as best as possible but it appears I'm running into a wall.
I'm getting an error when the code enters the Livebook.Session.Data.apply_operation/2
function during your script execution on Line 38. The error I'm specifically seeing is 1st argument: not an atom
.
At first I thought it was the .livemd file I was running the script on, but I've come to realize that it won't work even on just a super basic .livemd file that only has one code block returning an atom.
Hey @houllette! There was an internal change that broke it, I've just updated the script :)
In jupyter there is nbconvert for notebooks to other formats (python scripts included of course) https://nbconvert.readthedocs.io/en/latest/index.html
It may be nice to have something like nbconvert in elixir for livebooks
Bumping this idea b/c I think its great. These features would open up lots of potential new app ideas.
> livebook run ./playbook.livemd -x someval -y anotherval
Also a companion hex package would be great too.
output_livemd = livebook.run(file, params)
An app user story:
Applications: ETL, Security SOAR, if-this-then-that low code apps, any event driven system handling.
Livebook is great b/c it can bridge the gap between non-coders and developers. In the security industry as example, a developer could make an advanced livebook to run and enrich a security alert stream. Later a SOC analyst could investigate an enriched alert with an already-ran livebook and see pretty readable output with visualizations etc.
Sorry for the bump, for those coming from Jupyter Notebooks (plus to help SEO), this would be equivalent to the following command:
jupyter nbconvert --inplace --to notebook --execute example.ipynb
Anything more complicated than that requires papermill etc.
My use case is I could edit this outside the browser and rerun the workbook as I need.
However, IMHO, if you need to do this as a script, please write a mix task or consider something else :).
Add options to cli to inject input parameters at runtime and allow parameterized livebooks to programmatically execute a workflow/broadway multi-stage data pipeline.
Basically, allow livebooks with inputs (or exported scripts) to be used as-is in multi-stage broadway pipelines and pass values downstream.
Brent