Open dominykas opened 3 years ago
OK, I think this issue is on the critical path - the CLI approach is all nice, but wiby
needs to happen in CI.
I tentatively opened this referencing Github Actions, but actions don't make this easy, so I'll need to do some digging.
Kicking off the initial wiby test
is the easy part.
The hard part is to collect the results, because you can't trigger an action off a status change in another repo. Which means some sort of polling, but there's no obvious way of implementing that using actions either, or at least no obvious way which wouldn't use up the precious actions minutes.
There could be a scheduled which basically polls all the things that are in flight, but I'm struggling to imagine a way to make it easy to set up, as GH actions don't have a way of sharing workflows.
Neither is this easy on Travis - you'd essentially have to keep a job running until it times out or the dependents finish their jobs, which can be rather wasteful as well.
I'll ponder other available tools to achieve this, but ideas are most welcome!
OK, so here's the plan:
wiby test
, which will kick off the tests, and it will also set a status check to pending. You will then have to explicitly comment to wiby result
, which will loop through the tests that were kicked off and update the status check accordingly.
wiby install
would be awesome, but this also needs a silent way to just update the options. Maybe a wiby outdated
and wiby update
make sense. I'm thinking to do this as part of wiby
, but it probably makes sense to extract as a separate package as well, if the approach proves itself viable.Things not covered in #69, to be reconsidered if we want them for MVP:
details_url
does not do what I want it to do... Basically, the status check under the PR links to the details about the status check, but somehow does not want to link to the actual build details.wiby results
run. (I do hate the wiby results
comment pattern... but don't have any better ideas right now, that I can also implement fast, and not waste build minutes)wiby results
comment?wiby test
/ wiby result
comments?wiby
pushes to is based on the dependency name. We should do better than that and instead use the branch name of the dependency somewhere in there. This would allow linking multiple dependencies into one, e.g. you're testing dep-one
and dep-two
inside user-of-one-and-two
. Both dep-one
and dep-two
have a new feature developed on a branch named feat/new-stuff
. Running wiby test
in dep-one
would create a wiby-feat-new-stuff
branch with dep-one
linked into user-of-one-and-two
. Running wiby test
in dep-two
afterwards should see that wiby-feat-new-stuff
already exists and link dep-two
, while keeping dep-one
linked.
wiby results
check (indicating that the results check has completed).i am very interested in this “require action” functionality, and have never come across it before.
https://docs.github.com/en/rest/reference/checks - look for action_required
- the docs weren't super clear to me and I haven't had time to experiment with this just yet... And then there's a mention of actions
which allows "a button on GitHub that can be clicked to alert your app to do additional tasks", but it is unclear to me if you have to have an app or this is achievable for the workflows.
ah, that reads to me as just a way to indicate to github that the user has to click the "details" link, and do something else. visual diffing uses this, for example, to force users to go click a button confirming that yes, this visual change was intended.
Yeah, quite likely, although if that can be wired up to kicking off another job - that's all that's necessary. Or a readme telling to post a comment 🤷♂️
Looks like Github only sets the COLLABORATOR
relationship if you're added to the repo directly (seems like a known issue), but it does not create it when you're added via team (or by being an org admin). This means that the permission check for the comment is probably rather inefficient...
I guess we'll need to rewrite the comment trigger to instead be a label... which can kind of work for kicking off the wiby test
(the job should probably remove the label when the test ends), but there's still a need for a trigger to check wiby result
and I see no trivial solution (which does not waste minutes) to achieve this.
One option is to have a label for wiby in progress
. The collaborator of the repo would then remove the label to check for results. If the tests are still in progress - the job would add the label back, or something. Not a pretty flow, if you ask me... Comments are nicer :(
I guess I could rework this to check the permissions via the API, even if it would cost some actions seconds...
🤔
OK, the known issue with collaborators is still there, and it is a rather annoying one to work around, so I guess I'll try to reimplement the workflow with labels. Here's what I'm thinking right now:
wiby test
labelwiby
workflow:
wiby test
label from the PR
wiby in progress
label instead, just to make this status visible the PR list?wiby test
commandwiby test
labelwiby
workflow again:
wiby test
label from the PRwiby results
commandwiby
status to red or green on the commitGotchas:
wiby
status is pending, and downstream dependent builds are stuck for whatever reason - how does one kick off the wiby test
again? I'm trying to avoid multiple different labels (i.e. I don't want a separate wiby retest
or a wiby check results
label).Next steps:
wiby
status and wherever it is pending - adds the wiby test
label to kick off the wiby results
workflow
wiby test
(triggered off a comment?)