search

w3c / performance-timeline

Performance Timeline
https://w3c.github.io/performance-timeline/
Other
111 stars 34 forks source link

Auto-publishing not working #194

Closed yoavweiss closed 2 years ago

yoavweiss commented 2 years ago

It seems like auto-publishing is not working, and there are a couple of respec errors. Those 2 facts may be related.. (but would be good to be notified at PR time about the respec errors)

/cc @caribouW3 @marcoscaceres

marcoscaceres commented 2 years ago

Hey, so all PRs show if continuous integration fails. Click on any PR and you should see ❌ and the reasons why. For example: https://github.com/w3c/performance-timeline/runs/3770850099?check_suite_focus=true

for getting notified when publication fails, auto-publish supports adding an email: https://w3c.github.io/spec-prod/#w3c_notifications_cc

I can add you to the cc if you’d like.

yoavweiss commented 2 years ago

Ideally, I'd like the PR to not land if it has errors (similarly to how bikeshed based specs can't land if they don't build).

marcoscaceres commented 2 years ago

Right, that should be exactly the same already (bikeshed has this issue/feature also): What can happen is between the CI for a pull request passing and something gets merged the world has changed (e.g., the WebIDL parser changed, HTML removes a dfn, etc.).

In those cases, CI also prevents publication: why we are seeing an error right now. Something changed and ReSpec is telling us something is off.

I’ll take a look tomorrow.

yoavweiss commented 2 years ago

Cool, thanks!!

marcoscaceres commented 2 years ago

Ok, backtracking here.

Ideally, I'd like the PR to not land if it has errors (similarly to how bikeshed based specs can't land if they don't build).

This was already set up. In GitHub's settings, under branch protection:

Screen Shot 2021-11-25 at 12 57 24 pm

This is not a "bikeshed" or "respec" thing: that's controlled by spec-prod (the automatic publication system... and it was again flagging the right thing).

About the actual publication error, please always check the error that CI is emitting. In this case, it was:

Screen Shot 2021-11-25 at 12 59 47 pm

I.e., that the shortname was wrong: "performance-timeline-2-2".

This was cause by .github/workflows/auto-publish.yml: https://github.com/w3c/performance-timeline/commit/ca6936d3bcf5f0018d944fabed6e430b6d01b80e

@yoavweiss, I'm wondering if the problem is that the CI logs are too hard to decipher?

We have a bug to try to improve them, but I need to make sure we are not fixing the wrong thing... or fix them in the right way.

marcoscaceres commented 2 years ago

Just noting that auto-publishing is now working again ✅🙏

yoavweiss commented 2 years ago

This is not a "bikeshed" or "respec" thing: that's controlled by spec-prod (the automatic publication system... and it was again flagging the right thing).

OK, that's great to know!

@yoavweiss, I'm wondering if the problem is that the CI logs are too hard to decipher?

I haven't put in a ton of time trying to figure this out before filing, so can't say that better CI logs would've change it. I'll investigate more next time!

marcoscaceres commented 2 years ago

Appreciate it. And always happy to help. At the same time, we are trying to make the tools better so the community can best help themselves without relying so much on the Team 🙏