Publisher Queue Architecture

[Niicck]

Architecture Ideas: We can create a dynamically updating publishing queuing app with all the information we want. I'd estimate a sprint and change to do any of the first 3 options.

• [ ] Writing to json file in an S3 Bucket
• [ ] Writing to google sheets (Bob)
• [ ] Dynamically recreating an app
• [ ] Something with websockets instead of polling (a mystery)

Some Questions: Adding security is an extra 1/2 sprint to a sprint.

• [ ] Does it need to be secure? (non-public, only Joplin editors can view it)
• [ ] Do publish requests made be different users need to be secure? (impossible for other Joplin editors to see)

[easherma]

I think creating a Django app inside of Joplin would be a quick and easy route to success for MVP. Wouldn't any of those first three options would still need time and additional work to set up messaging between it and publisher and joplin? We’ll have handle sending the publish signal in joplin anyways, that can be a message that gets stored in the DB and then handled for how it gets added to a queue. Receiving additional messages in joplin then isn't much of a leap.

The publisher is going to need to message out for some kind of status (building, in progress, done) which we could set up an endpoint in joplin to receive.

Polling would be fine for MVP, as our builds take many minutes right now anyways so even if we polled every 30 seconds it would be fine.

Post-MVP, Wagtail now supports Django 3.0 which supports websockets natively so we could refine the endpoint further.

This is a pretty decent but simple example:

https://stackoverflow.com/questions/56913676/dynamic-updates-in-real-time-to-a-django-template

It uses Django REST Framework, which we already have. Or we could use graphene and have an endpoint there for page build status with some mutations for updating that status.

That would make handling security/authentication more at hand, already give us a database and api to write data into(and have a record of builds), and have the information already at it’s intended destination.

Any thoughts on that?