Twitch integration: Channel point rewards to trigger arbitrary events for the VR streamer (affiliate)

[BOLL7708]

What would this gadget do?

Allow a Twitch viewer to use the built in channel point rewards to cause arbitrary events to happen for the VR streamer.

Who would use this gadget?

Twitch streamers that want to give their viewership a means of interacting with the stream in a tangible way that is also well integrated into the Twitch platform.

Assuming that you're on the team, what other skillsets would you need to make this project happen over a couple days of hacking?

As I'm used to work on my side projects alone I don't really expect anyone else to jump in on this, I'm also in Europe so it might affect who could help out if we need to communicate. Myself I will probably manage the Twitch integration, as I'm a backend person at my day job. I have used TypeScript but never looked at React before this, so React knowledge would be nice I guess.

What will be the toughest part of building this gadget?

It starts with the integration against the Twitch PubSub API, which appears to be an active WebSockets connection, which I have experience working with. It needs an OAuth access token with specific scopes for the API access, so will also need an OAuth authentication before anything can work, I'm hopeful we can use the OAuth2 Device Flow for this, which is available for Twitch on consoles today, but I'm not sure that is open for anyone. The authentication will be something the project hinges at, and might be something I will have to explore beforehand, if there's time.

Other than that, React is an unknown to me as of yet, but from what I've seen on the Slack server it should be fairly straight forward. I imagine setting up a few example events that can be triggered, so hopefully not too complicated, but it could be nice to add a configuration gadget that would help with setting it up but I imagine most of that to be done on the desktop.

Side notes

Something that pushed me to submit this was hitting Affiliate on Twitch, not a huge deal, but that unlocks a few platform features of which Channel Points is one. As it is I have infinite channel points for my own channel, so I can most likely test this out without hassle. It is possible to define a range of custom rewards for channel points which is why it's a suitable function to use.

[JoeLudwig]

Seems like this would be a room gadget, which would let the twitch viewer and streamer see each other's gadgets. Is that right?

[Adil3tr]

Do you see this as just transitioning the 2D UI of twitch into VR to give a good spatial AR representation like a watch or an animated model that pops up with the message and 3D confetti appearing in the steamers view, or adding something specific to the viewer contributing? Because say that it would let users spawn a model, and I'm wondering if it's supposed to be adding something to the game environment, but would move with the player since it's tied to his room-scale, or adding something to the user's experience like a parrot on their shoulder or giving them goggles, something relative to their body. Am I understanding the concept?

[BOLL7708]

@JoeLudwig The viewer would really be someone on the Twitch webpage or in the Twitch app, so there would be no direct gadget sharing going on, even if that sounds kind of neat as well. The intent is to connect to base facilities of Twitch to make any viewer capable of using the feature.

It being a room-gadget sounds about right, the idea is to have a viewer convert channel points, which they get from watching, to place whatever model somewhere in a random place in the play area as a session-permanent decoration. See the image below for how this interface looks like right now with the default options, these can be replaced with custom ones and it is my hope and expectation that those are communicated over the PubSub interface. There would still be a use for a streamer-gadget, but mostly for setup. I imagine something that would display the device flow authorization code, for the access token, and perhaps some administrative buttons for whatever needs to be done. This should be hidden when actually streaming though.

@Adil3tr Adding things like chat would not be part of this specific gadget, for the moment, we would connect to the PubSub API mostly to get channel point events but also bits if there is time. The idea with the model spawning is that it will be, as is alluded to in the description, not really something that helps the streamer. Rather the opposite, it is a way for the viewer to directly affect the VR experience, probably negatively, but adding something to the play area. As the interface should just be "This was paid for" the plan was to place it randomly around the play area. The models would be static to the space, meaning if smooth locomotion is used, they will seem to slide around, which is weird and they can also cause depth disparity, again motivating the sabotage description. 😅

[Adil3tr]

Okay, I was curious because you said the project "only wants to place objects randomly or rain particles, might not be overly complicated from a 3D scene generation view point." If you've played Astrobot some enemies target the players head and hit them with projectiles that obscure their view (even though it's a third person game). I can imagine that could get quite fun if chat can spawn something, maybe along the room edge so it's visible and gives them some time like it's a nintendo style cannon that pops up somewhere on the line and fires an object determined by donation size, for the streamer to dodge. It would encourage dogpile donations to make it harder since it's along a line and so there is no fixed number of spawn points.

[BOLL7708]

@Adil3tr that might actually be an even better idea, spawn something in front of the VR player that will shoot at their face, hah. I think the base for this project will be getting:

1. Authentication working
2. API connection to Twitch
3. Trigger events inside Aardvark from points use

So I guess a lot of it is just getting API connections working, maybe I should do that beforehand so the hackathon itself is focused on Aardvark development specifically, adding the effects and results of triggering arbitrary events. And if there's time, add a configuration gadget to go along with the first time setup, which could end up becoming the reusable part of the project 🤔

[BOLL7708]

Ah, derp, at least according to this old post the device OAuth flow is not actually available for just anyone, there's not even a way to apply for access? That's a disappointment and removes the dead simple way of getting an access token.

[JoeLudwig]

I'm a little concerned that the audience size for something targeted at the streamer side will be small. Maybe it's a useful niche though?

[BOLL7708]

Oh yeah this is something maybe only me in the whole world needs or wants, but what I imagine this being is kind of what drives my motivation as well. I'm a tiny almost non-existent streamer, but I've spent a lot of time making integrations in other ways, like piping Twitch chat into the SteamVR notifications and a custom overlay including data acquisition tools. I guess I might find more entertainment in making the actual integrations than actually streaming, hmm. 🤔

Due to it being so niche I don't expect anyone else to join this specific project, and I don't expect it to be completely finished at the end either, but a working prototype that can be demoed at least. Maybe that goes against the spirit of the hackathon, now I'm not sure, I should spend more time reading the documents.

Honestly my weeks have been quite booked lately so I'll be slipping into this on a banana peel. We'll see how it goes. Maybe I should save this for after the hackathon and look at some of the other suggestions I have seen as useful, like a stopwatch or taking notes.

[Adil3tr]

I was watching a twitch steam yesterday and I think his entire donation system runs through Streamlabs on top of Twitch,or something? Is there a third party solution, especially one of the ones that gives creators a better cut, that has an easily usable api? It seems like a viable option.