Project idea: The Peanut Gallery (and a call for participants)

callaa commented 7 years ago

I've received many requests for features that would be useful for certain use cases, but which I feel wouldn't fit well into the application itself, such as music players or gaming related utilities. There are also some features that would be nice to have, like built-in live streaming, which would be difficult to implement.

So, I'm presenting an idea for a new project I'm calling "peanut gallery", that would provide a companion web site for a Drawpile session.

This site would serve two purposes:

It would provide a way to spectate and comment on a session without having to join it.
People logged in to a session could use it to gain extra features, such as a shared music player and whatever new stuff we'll think of in the future.

Here's a rough mockup/wireframe (just a first draft, not a final design by any means):

peanut-gallery

Usage

The user goes to watch.drawpile.net and is presented with a list of Peanut Gallery enabled sessions. They select a session and enter a username. A password too, if the session is password protected. They are then logged in to the Peanut Gallery and can watch the live stream, comment in the chat box and use all the other features.

Note: the peanut gallery is a server specific feature. The one at watch.drawpile.net could only be used for sessions hosted on the public server. For other sessions, the person hosting the server must also host their own copy of the Peanut Gallery software.

Features for the first version:

Password protection: if the session is password protected, the same password should be needed to access the peanut gallery as well. An option to allow anyone to spectate would be nice too.
Live stream: spectate the session without joining (not necessarily full resolution and no layer access. Must be possible to switch off)
Chat integration (can be blocked in the session)
Some kind of admin controls are needed to block hecklers

Technical architecture

pg-architecture

Backend server provides live updates to the fronted using a Websocket based API
Backend connects to a session like a regular Drawpile client (does not need to live on the same physical server)
Will probably need some serverside support, e.g. to invite/block PG access and grant access to PG specific commands
Front end will be a JavaScript Single Page App (alternative frontends, such as an Android app, can use the same server too)
Plugin based architecture: new features should be easy to add

The biggest architectural question right now is how the live stream feature should work. Possible approaches:

Live stream is hosted on a service like picarto or twitch by some member of the session: requires lots of manual work to set up every time, likely to be an endless source of compatibility problems.
Live stream built into the PG server: a clean all-in-one solution, but requires the server to be written in C++. May expose security vulnerabilities and a crash will take down everything.
Live streamer is a separate module (written in C++) which is run by the backend server.

So far, the last option seems best. The live stream module has to be written in C++ so it can reuse the Drawpile paint engine. (Rewriting it in some other language while ensuring it works exactly the same would be a massive undertaking.)

Separating the livestreamer into its own process provides some extra safety and means the PG backend can be written in a more suitable language. (Also, I'm planning on creating a headless benchmark tool anyway, might as well make it a livestreamer while I'm at it.) It also makes it possible to support multiple protocol versions, by having different livestreamer modules for each one.

Another question is the livestream format. WebM is one possibility, but a simple custom format might not be a bad choice either.

For example, the format could be as simple as a stream of messages like this: {x, y, JPEG data}. Each message would paint the included JPEG image at the given coordinates. This would fit well with the way Drawpile works as, unlike video, most changes are either confined to small rectangles, or (more rarely) update the entire image. Occasionally, the livestreamer would send out an "I-frame" that repaints the whole screen so new users can catch up and any accumulated errors are cleaned out.

Call for participants

This project is a bit too big for me to handle alone, so to make it happen I need help. Specifically, I'm looking for:

Front end developers who know a JavaScript framework such as React
Backend developers who know Go

After rewriting the listing server in Go, I feel the language would be suitable for this project, especially since goroutines and channels make it easy to work with web sockets.

KnicKnic commented 6 years ago

With QT & web assembly getting a little further along, you may want to consider that. http://blog.qt.io/blog/2018/05/22/qt-for-webassembly-examples/ . I am not up with web assembly (I do have a future project I want to tackle with it.) One big issue I know of is threading support https://github.com/WebAssembly/threads

askmeaboutlo0m commented 1 year ago

(I've taken over development on Drawpile, so I'm going through these tickets and organizing them.)

This project exists now in the form of drawdance-web: https://github.com/drawpile/Drawdance

drawpile / Drawpile