Setup ci

[mbruns91]

Here, I setup the CI for the automatic (headless) execution of the workflow. (heads up: with "workflow" I'll reefer to the actual pyiron workflow. When I'm talking about a github workflow, I'll call it "gh-workflow".)

The idea is like so: A team wants to collaborate on a workflow using pyiron_workflow. Some devs are now working individually on the required nodes. As soon as there is a (maybe dummy) workflow existent, the team can open a repository following this repo as a template and include their nodes (by copying them to ./nodes/). Now the create a branch to work on, and all devs are committing their work. When the label "run_workflow" is added, each commit then triggers a headless execution of the related ./workflow.ipynb using jupyter nbconvert. The changes made to the notebook workflow.ipynb (e.g. printed results, shown figures, the drawn workflow, ...) are written to a notebook called workflow-out.ipynb. This is then uploaded as an artifact and can be viewed looking at the respective actions run. This way, a dev can see the changes caused by her commit in a rendered form.

In a first draft, I hard-coded some stuff that could be left to a user to be defined (e.g. the name of the conda environment or the name of the workflowfile/notebook). However, once the repository's CI works as intended, we should merge this PR. I will then improve the whole usage in another PR.

[mbruns91]

I think this is working now. The only thing I don't like, is the fact that you cannot view the notebook after the gh-workflow run in the browser. At this point, you have to click on "Actions" and select the run you're interested in. There, under "Artifacts", you can download a zip file, which contains the updated notebook, which you can view locally.

I don't think we can do anything about that, as this is partly related to the Github UI. There is an issue where in-browser view of certain files uploaded as artifacts is requested, but it's still open.

[samwaseda]

Ahaaaa that means workflow-out.ipynb cannot be displayed, correct? That indeed sounds a bit like a party pooper. Does @liamhuber maybe have an idea?

[mbruns91]

Ahaaaa that means workflow-out.ipynb cannot be displayed, correct? That indeed sounds a bit like a party pooper. Does @liamhuber maybe have an idea?

That's exactly the issue.

A possible solution is that we could allow triggering the CI not on each push to a branch but to also be supressed (somehow). Then, the workflow-execution-action could re-push the updated notebook to some directory ci_outputs/ nd it would be viewable in the browser. For public repositories this is rather straight forward. For private repos you have to handle access tokens, but it is possible to do that.

[liamhuber]

Ahaaaa that means workflow-out.ipynb cannot be displayed, correct? That indeed sounds a bit like a party pooper. Does @liamhuber maybe have an idea?

There is a way to customize the output in the actions report, but I don't even have a link to the docs for you right now. So I imagine you could write a little customized report from your notebook, e.g. the target output or something, so you only have to look at the actions instead of the artifact. Or maybe you can even do something fancy like embedding an image and somehow render and embed the notebook? But I'm not actually sure that's possible.

[liamhuber]

Possible tool: https://github.com/marketplace/actions/codefresh-report-image