search

cabal-club / cabal-client

interface for writing peer-to-peer chat clients
GNU General Public License v3.0
97 stars 14 forks source link

Implement audio/video streaming #60

Open garbados opened 4 years ago

garbados commented 4 years ago

As demonstrated by hypervision it is possible to stream video over hypercores. This has wondrous implications for a chat application like Cabal, as it makes possible group voice chats à la TeamSpeak, and video chats à la Zoom or Jitsi, all P2P.

See also: https://github.com/cabal-club/cabal-client/issues/15

garbados commented 4 years ago

Per https://github.com/cabal-club/cabal-client/issues/15#issuecomment-651986847 I suggest the use of two new message types: chat/video and chat/audio. Each links to a hypercore into which is being streamed a media feed, such as someone's webcam or microphone. When the writer of this feed is online, the feed is considered online -- that is, if I am peering a hypercore of my mic, a cabal client might display a mic or headset next to my name, indicating that I am listening to the channel's aggregate audio feeds.

I am not well-versed enough in audio and video streaming to outline a robust message schema with appropriate annotation for the involved media. At this point I can only say, confidently, that it is possible.

okdistribute commented 4 years ago

Awesome idea!

I think there are a couple different user affordances here...

1) persistent audio and/or video chats... this is great for accessibility! We could limit the size if we wanted to, also, similar to WhatsApp/Signal voice messages. This would be well implemented in the same feed as images, for example as a separate content hypercore, how hyperdrive works. 2) live audio/video chats... this is could be implemented as a webrtc hook for ephemeral audio/video without needing a separate hypercore, but the chat message could still be in the chat feed as chat/live, and with the metadata necessary to join the live chatroom.

hackergrrl commented 4 years ago

This would be very cool.

I think it could make sense to split this off from core cabal: a peer-to-peer video & audio chat platform would be very useful in a variety of contexts, even outside of cabal. From there, the question would be how cabal wants to integrate with it, which could be very simple chat/meeting messages that indicate an AV room's existence, or even the beginning & end of a time-specific call.

garbados commented 4 years ago

Fair point. I'll see what I can do @noffle

okdistribute commented 4 years ago

Yeah @noffle this is what I was thinking too for #2 -- webrtc seems like a good first start for the Desktop app -- is it available in Electron already?

I do think #1 would be super useful as well (but could be accomplished with filesharing in cabal, so perhaps is not related to live meetings)

hackergrrl commented 4 years ago

Oh sorry @okdistribute I somehow missed your comments where you basically said the same stuff. >__<