veqtor
commented
7 years ago I think VJ'ing is best implemented as a per-parcel script, remember, this is VR, so we as VJ's/Light designers wouldn't have to follow conventional physical rules, making something too specific will only limit the options, so I would like a very open-ended scriptable system. But things like kinect- or other depthsensor - streaming of point-clouds would be great.
As for audio streaming, it would be best to simply support common web standards for video and audio streaming such as dash & hls
CodeSchwert
Name: Live Audio Streaming
Purpose: Stream music from live audio performances such as DJ’s live set in studio (or venue), live bands, studio sessions, to a VR venue in Decentraland. This could be used to extend live performances into the VR world allowing VR audiences to experience events tied to the real world. Or, it could also be used for musicians to perform in VR only events for Decentraland audiences.
Description: Imagine turning up to a venue in Decentraland, like a nightclub or music hall, but more fantastical since it’s in Decentraland, to listen to a live DJ, or band, or pianist, or some other audio performance. There might be a VR representation or video feed of the performers, or maybe just a screen with visualisations controlled by a VJ on several of the venue walls. The performers start to play their live set, but they can see the punters of Decentraland on screen, and are able to chat/message them, and thus can adjust their set according to how the crowd reacts.
A proposed system for streaming live audio to a Decentraland tile, depending on the messaging system built into Decentraland, requires a method for allowing performers and users to interact. The components for live audio streaming performances:
The main components are going to be the client on the performer side that takes an audio channel/output and streams it to the receiver client on the Decentraland side. This streaming could be achieved with WebRTC. So, the clients would be WebRTC peers communicating directly with each other, or more precisely, many Decentraland peers connected to the performer “broadcasting” an audio output.
The performers WebRTC client could be a desktop app built with Electron.js. Performers might be able to select which audio device / channel output to use for sending audio to Decentraland. The Decentraland receiver client, which would essentially be a WebRTC wrapper, would need to be packaged into the content for the “venue” plot of LAND.
A caveat for running a WebRTC system would be the requirement for a signalling server to establish the initial peer connections. This cost might be covered by user attending the VR events/venue. Users could pay a small “door” charge to cover the costs of the server for the duration of the event.
The system might work like this (very high-level overview):
The optional extra features of video feeds / VJ feeds would completely be optional to begin with, but as the VR live streaming events mature, might become more desirable.
In the future, this system could also be modified slightly to be used with any kind of audio streaming situation, like, teaching interactions, conferences, debates, etc.
To begin with the tiles required for the “venue” would be minimal. A single LAND tile might suffice. The main concern would be to get the WebRTC clients working first. If live audio performances in VR really takes off, then more LAND might be required to expand the “venue”.
The performer/user clients would be written and maintained by the community, allowing for anyone to use the software and be able to setup a audio receiving venue.
Decentraland Slack Username: maelstrom.tech