search

cityofaustin / techstack

Project management for the City of Austin's new digital service delivery platform, Austin.gov.
11 stars 3 forks source link

Pages stuck in "Publishing" #5024

Open gabrielaperezdelaosa opened 3 years ago

gabrielaperezdelaosa commented 3 years ago

Describe the bug

A series of pages in the staging environment have gotten stuck in publishing status. It does look like the published/updated, just Joplin does not reflect that.

URL

https://joplin-pr-demo.herokuapp.com/admin/pages/search/?p=2

To Reproduce

I don't know why it happened. I will say that I went through maybe a dozen pages and added a contact snippet and "updated" rapidly. So maybe that's what "broke it"? But the stuck pages are not just the ones that I updated.

Screenshots

image.png<!--- If applicable, add screenshots or a gif to help explain your problem --->

Device and browser

PC - chrome

Niicck commented 3 years ago

Ok, this is a fun bug. So this will happen if a publish request is "cancelled" by another build. A publish request will be cancelled if the same Janis is built by a different Joplin.

Ex:

  1. Someone published page 933 on "demo" Joplin to be published on "demo" Janis.
  2. That publish request is submitted to the Publisher.
  3. For reasons unrelated to this issue, that build fails.
  4. That original Publish Request goes back into the queue. It's still a publish request to build page 933 on "demo" Janis using data from "demo" Joplin.
  5. I have merged code into "demo" Janis that will fix the build breaking.
  6. However, that github-triggered Publish Request was set to build all pages on "demo" Janis using data from "staging" Joplin, not "demo" Joplin.
  7. Here's what publisher does. When it sees that a Publish Request wants to build on the same Janis using a new Joplin, it will cancel all of the build requests that are not from that same Joplin. "demo" is not "staging", so the old "demo" requests are cancelled.
  8. That leaves the publishing statuses on "demo" Joplin in limbo.

To solve MVP: I will manually update the unsynced publishing statuses on "demo" Joplin.

To solve for real: You'll have to change the logic of how Publisher handles build requests from 2 different Joplins on the same Janis.

This problem is unlikely to happen in Production, since Production Janis can only be built by Production Joplin.

But it's causing issues for important PRs. It would be good to fix.