Display the updated notebook after each gh-workflow run

[samwaseda]

I really like the idea with the label!

[mbruns91]

I really like the idea with the label!

Thanks! Also, if you do git config user.name 'ci', before committing/pushing, the gh-workflow won't be executed. This is mainly for in the next step pushing the updated notebook with this user name back into the repo, such that it can be viewed online. Otherwise, each gh-workflow run would make a push and thus trigger the ci and we enter an endless loop...

And - of course - this can be used as a workaround if someone intentionally wants to make a push that doesn't trigger the ci. But I'll come back to this later and see if I find a better way do achieve that.

[samwaseda]

I expected to see ci_output/workflow-out.ipynb but that's not the case here?

[liamhuber]

I expected to see ci_output/workflow-out.ipynb but that's not the case here?

It got stored successfully as an artifact during an earlier successful run of the workflow: https://github.com/mbruns91/graphathon/actions/runs/9565869540

What would be cool next is to auto-generate a comment in the PR that links to some render of the same notebook. I thought at first we could just pass it to nbviewer, but the notebook is zipped so passing the artifact straight to the nbviewer site fails with a 404. Still, there is a possible attack route here where we automate the download of the artifact, push it somewhere we can see it unzipped, pass that link to nbviewer, and then post the nbviewer link in the PR. Maybe we can find a more direct solution though.

[mbruns91]

I expected to see ci_output/workflow-out.ipynb but that's not the case here?

Yeah, I did not figure out the correct way to do the push yet...

I expected to see ci_output/workflow-out.ipynb but that's not the case here?

It got stored successfully as an artifact during an earlier successful run of the workflow: https://github.com/mbruns91/graphathon/actions/runs/9565869540

What would be cool next is to auto-generate a comment in the PR that links to some render of the same notebook. I thought at first we could just pass it to nbviewer, but the notebook is zipped so passing the artifact straight to the nbviewer site fails with a 404. Still, there is a possible attack route here where we automate the download of the artifact, push it somewhere we can see it unzipped, pass that link to nbviewer, and then post the nbviewer link in the PR. Maybe we can find a more direct solution though.

I really like the idea of posting a reference to a rendered version of the notebook in the pr! I'll figure out a way to have something coming as close as possible to seeing it directly in a posting.

[mbruns91]

Ok, now you can see the updated notebook in-browser by navigating to ci_output/workflow-out.ipynb on the brach the PR is opened from after pushing!

One important note: to avoid divergent branches on local and remote, one now has to wait for the action to terminate and then do a git pull to receive the push done by the action. I will find a way to avoid this as it is super prone to be forgotten. And - at least for long-running workflows - it is kind of inconvenient having to wait with continuing development.

[liamhuber]

@mbruns91, awesome! I think I know how to get the workflow to post a comment in the PR containing the URL to the executed notebook so it's just a click away. Before merging here, please let me play with that on one of my repos. I'll either post a snippet for you to past into the workflow or let you know if I've failed :joy:

[liamhuber]

🚀 🚀 Working A-OK in my trials over on pyiron_snippets!

Try appending this step after the push ./ci_output step:

      - name: Post a comment linking the executed notebook
        uses: actions/github-script@v7
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          script: |
            const issueNumber = context.payload.pull_request.number;
            const branchName = context.payload.pull_request.head.ref;
            const repoUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/blob/${branchName}`;
            const relativeTarget = `ci_output/workflow-out.ipynb`;
            const comment = `:robot: Here is the [workflow output notebook on this branch](${repoUrl}/${relativeTarget})`;
            await github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: issueNumber,
              body: comment,
            });

I am super happy how fast this was. Just to share the magic sauce here, the github API available via actions/github-script@v7 is extensive and sensible, and I got ChatGPT (4o) to give me a passably-close attack so I didn't need to have all the syntax memorized. (The one catch is it has an outdated syntax for the actual API call, and that would have taken me a while to solve today, except I already dealt with it last month upgrading pyiron/actions to @v7 so it was easy-peasy.) I definitely recommend skimming the API docs for getting a rough sense of what is possible, then leaning on the bot to help with syntax.

We could think about being more sophisticated, e.g. there's no need to re-post the link on each run of the workflow and currently we do, but it's not an expensive call so I'm not stressed about that sort of fine polish atm.

[samwaseda]

So now we are talking about the links to the artefacts? I think it's good that the final notebook is uploaded directly to the repository, no? At least as long as we don't have conflicts.

[liamhuber]

No, it's a link to the uploaded artifact in the ci output folder

[samwaseda]

No, it's a link to the uploaded artifact in the ci output folder

That's not the same thing?

[liamhuber]

The artifact is a zipped notebook and a link to it probably triggers a download, while the artifact uploaded to the branch is a file and a link to it takes you to the repo page for that file, which renders it nicely in your browser like any other notebook in the repo.

Go look at the snippets pr I linked to see a demo

[liamhuber]

No never mind, the snippets demo is broken because I already closed the pr and deleted the branch. Just add the code I gave here and see how it works 😝