Server components

[benhylau]

OBS device (on-premise)

• Takes video and audio feed
• scp ts chunks to multiple IPFS publishing servers (for redundancy)

IPFS publishing servers (data centre)

• High bandwidth uplink and well-peered into IPFS network
• ipfs add ts chunks received from on-premise device
• ipns publish m3u8 manifest referencing ts chunks
• Request ts and m3u8 from public HTTP gateways to warm their local IPFS cache

Dedicated HTTP gateways (data centre)

• Requires "unmetered" bandwidth cap
• Most people would be watching the streams through these gateways, unless they run a local IPFS instance
• Run one or more dedicated HTTP gateway throughout the conference (so we can ensure sufficient resource allocation as public gateways often get congested)

Archival servers (data centre)

• Does not require "unmetered" bandwidth cap
• Permanent servers to fetch and pin all video assets for archival purpose

Some of these could be physically the same server.

@ASoTNetworks you mentioned chunking could be done in the IPFS publishing servers? Is this consistent with you have in mind?

[darkdrgn2k]

chunking could be done in the IPFS publishing servers?

I would Chunk at the source before distribution. This way when added to ipfs they will be the same hases (ie no dropped udp packet). Otherwise there would be two different cimplete set of feeds depending on the m3u8 used

[benhylau]

Adding details about the OBS device (on-premise):

+------------------+             +-------------+            +--------------------------+
| HD Video Camera  | --(hdmi)--> | Elgato HD60 | --(usb)--> | Laptop running OBS       |
+------------------+             +-------------+            | - Streams to RTMP server |
                                                            |   for Internet stream    |
+------------------+             +-------------+            | - Records m3u8 + ts      |
| Presenter Slides | --(hdmi)--> | Elgato HD60 | --(usb)--> |   for IPFS stream        |
+------------------+             +-------------+            +--------------------------+
                                        |
                               (hdmi pass-through)
                                        |
                                        v
                                    Projector

*Make sure two separate USB buses on the laptop are used to avoid saturating the bus bandwidth

• The client video player will play both RTMP (Internet stream) and HLS (IPFS stream via HTTP gateway)
• The RTMP server is remote and runs: https://github.com/arut/nginx-rtmp-module (need to sort out auth)
• HLS .ts chunks for IPFS publishing and local archiving are generated by OBS (@ASoTNetworks @darkdrgn2k how do you want to get these to the IPFS publishing servers? scp/sftp? Or do you intend for these servers to pull from the RTMP stream and chunk there?)

[ASoTNetworks]

My plan is to have a IPFS publishing server pull from RTMP and it will chunk using this command ffmpeg -i rtmp://server-hostname/stream/key -c copy -hls_time 10 LIVE.m3u8

On that IPFS server we can also SFTP / SCP to other IPFS enabled servers to spread the load.

[benhylau]

So there is no need to locally record HLS chunks at all? The only thing the local machine uploads is the RTMP stream, pulled by both Internet server and one IPFS server?

[darkdrgn2k]

That works and it'd probobly the easiest way of splitting the stream to multiple servers.

But I will point out that your adding a new single point of failure

The obs system is always one (if it's down no stream)

But now the rtmp service is now a second.

[benhylau]

After further discussions, OBS will be recording .mp4 files for local archiving, and streaming RTMP to be picked up by HTTP streaming servers and IPFS streaming servers. @ASoTNetworks will consume the RTMP stream on his HTTP HLS streaming servers (non-IPFS) and the IPFS servers will take that same stream to do all the IPFS stuff.

This repo will focus on the IPFS side, and leave what people want to do with the RTMP stream as an implementation detail. So the updated OBS device (on-premise) picture looks like:

+------------------+             +-------------+            +---------------------------+
| HD Video Camera  | --(hdmi)--> | Elgato HD60 | --(usb)--> | Laptop running OBS        |
+------------------+             +-------------+            | - Streams to RTMP server  |
                                                            |   for HTTP & IPFS streams |
+------------------+             +-------------+            | - Records mp4 files       |
| Presenter Slides | --(hdmi)--> | Elgato HD60 | --(usb)--> |   for local archiving     |
+------------------+             +-------------+            +---------------------------+
                                        |
                               (hdmi pass-through)
                                        |
                                        v
                                    Projector

[benhylau]

Mostly addressed in README but still unsure whether we will use Justin's server (with unmetred bandwidth) for ipfs mirror/gateway.

[benhylau]

We will use Justin's server as a mirror. Since README has everything documented and this ticket has outdated information, let's close this and refer to README.