search

maxheld83 / ghpages

Deploy arbitrary static assets through GitHub Actions
https://github.com/maxheld83/ghpages
MIT License
179 stars 31 forks source link

run integration test #10

Open maxheld83 opened 5 years ago

maxheld83 commented 5 years ago

right now, the testing is kinda informal: you can just go to http://www.maxheld.de/ghpages/, and ideally, you should see the last commit sha there if things worked out.

I'd like to have a more fully programmatic way of testing this action, especially as more people rely on it, but I haven't quite figured out an elegant way to do this.

I'd probably involve talking with the GitHub API somehow, to check that the action (given some boilerplate setup) works as intended.

I am also developing this as part of the R package ghactions where I have good unit/integration testing tooling, and I'm not quite sure how that would work here.

Maybe postpone this to a later date, maybe github comes up with something.

maxheld83 commented 5 years ago

also relevant / same problem: https://github.com/maxheld83/ghactions/issues/22

maxheld83 commented 5 years ago

just asked github about it, see what they'll say:

I'm starting to wonder how to best (integration) test github actions.

What I mean by integration tests here is that I want to make sure that any given commit in the action repo (say https://github.com/maxheld83/ghpages) actually completes the intended action (or fails in expected ways) given some boilerplate repo on which to run the action.

My initial thoughts where that this would require me to talk with the GitHub API, maybe fork/init some boilerplate repo, commit the a .github/main.workflow with the action call in question, and then poll the API to see whether the check failed or succeeded. The result of the API poll would then, in turn, be a result in the .github/main.workflow of the original action.

I guess one way of thinking about this is that I'd need an action to test actions.

I could do this in a language-specific way, using the integration/testing tooling I have on any given platform (say, R in my case), but it seems a little awkward.

I am aware that you guys have open sourced the parser now, but I'd want to go one step further and actually test against live GitHub actions (your service).

This might seem a bit obscure, but I've always thought that testing the CI/CD setup is really important. Nothing more frustrating than have some downstream project fail, and not knowing whether "it's you or the CI/CD setup".

Do you guys have any leadership/guidance on this already?

Any chance you could build an action to test actions in a language-agnostic way? And/or could we somehow package boilerplate repos with the actions?