Question: Multiple YT-synch services?

[bedeho]

Question

Should there be one or multiple YT-synch services?

One

There could be one service, that is one backend, auth and replication service, which can be used from multiple applications simultaneously. We could even make sure to have proper credit assignment in allowing to capture the member representing the app when adding creator to the backend, which can be used to do other reward payments later. Of course nothing actually prevents someone from running a new synching service, however, unless the tooling is properly setup for that, it is unlikely to work that well, and thus be attempted in the near term. For example, as a creator tries to use more than one app, each synching services would be unaware that the same channel is already created, or worse yet, they may start replicating videos into the same channel. This does not need to be malicious on the part of any app or creator. A malicious creator could attempt to do so in order to get duplicate rewards, depending on how rigorous the verification is at any given time, which depends on how well verification scales. Another benefit of one synching service is that it likely will give creators a better experience, as resources in operating that service properly in terms of support, cost, abuse detection/prevention, reliability and communication with creator will be better than lots of distinct services with limited resources and expertise. It will be easier to also apply fixes, migrations and bugfixes, without having to coordinate. The proper use of category mapping from YouTube to Joystream, which is still an open question(#38), will also be easier to iterate on. Lastly, since most of the early gateways will be operated by workers of the gateway working group, they will all be in proximity with the lead and the rest of the DAO, hence there is just less of a need for someone to run their own distinct operation.

Multiple

The main benefit I see here is that new operators dont need to coordinate with other participants in any way at all, and it is more permissionless. There is also some extra level of fault tolerance perhaps.

[dmtrjsg]

From my pov we want the GW operators to have the smoothest journey on the way of scaling their app. Seems like that is achieved with "One". After having read both descriptions I do not see any drawbacks of starting with one.

"they may start replicating videos into the same channel"

We want to avoid this as there's no easy way of reversing this rn afaik

"A malicious creator could attempt to do so in order to get duplicate rewards"

Are all rewards for all GWs paid out of GS Jensis pot? I assumed the operator would run their own incentive scheme for it, but we would only provide the infrastructure to track the stats used in defining the rewards..

Overall considering ease of setup and maintenance it does seem like starting with One is more fabourable.

[dmtrjsg]

From my pov we want the GW operators to have the smoothest journey on the way of scaling their app. Seems like that is achieved with "One". After having read both descriptions I do not see any drawbacks of starting with one.

"they may start replicating videos into the same channel"

We want to avoid this as there's no easy way of reversing this rn afaik (for creators)

"A malicious creator could attempt to do so in order to get duplicate rewards"

Are all rewards for all GWs paid out of GS Jensis pot? I assumed the operator would run their own incentive scheme for it, but we would only provide the infrastructure to track the stats used in defining the rewards..

Overall considering ease of setup and maintenance it does seem like starting with One is more fabourable.

[bedeho]

Are all rewards for all GWs paid out of GS Jensis pot? I assumed the operator would run their own incentive scheme for it, but we would only provide the infrastructure to track the stats used in defining the rewards..

Just to clarify, GWs likely should in the early days be hired, so their rewards are paid by the DAO, this is distinct from rewards to creators signing up through a specific GW, which I think is what your question pertains to.

I think that it does not make economic sense to require GWs to invest in incentivising creator adoption when that generates big spillovers in the DAO which they cannot capture well. The DAO wil able to justify a much greater scale of investment in this. I think that we, as Jsgenesis, are simply doing this in place of the DAO until

a) My Payments work b) the DAO itself adopts it as a consistent long term policy: this may happen very soon, I don't know, but literally on day 0 who knows.

[kdembler]

If there's one synching service, how does that work? Who is it operated by? Who holds an ownership over the collaborator used to execute uploads? This requires trust for whoever operates this service. I think our goal is to build tools for people to be able to run their own apps, independently, and not rely on third party to both continue running a service and not be malicious.

For example, as a creator tries to use more than one app, each synching services would be unaware that the same channel is already created, or worse yet, they may start replicating videos into the same channel

True, but this sounds like an edge-case for me, one that could probably be easily solved by some kind of a convetion-based signal.

I'm also unsure how incentives would work if there was a single YPP backend, who would be paying for those?

[zeeshanakram3]

but this sounds like an edge-case for me, one that could probably be easily solved by some kind of a convention-based signal.

Do you mean something like each syncing service has a well-defined responsibility of only syncing channels/videos belonging to certain category/s? Seems like a doable thing to me. But we still can't enforce it

[kdembler]

I was rather thinking about some way to mark that a channel has a youtube sync enabled already. For example, collaborators responsible for sync could have a special member name or something like that (super simple example)

[bedeho]

Who is it operated by? Who holds an ownership over the collaborator used to execute uploads? This requires trust for whoever operates this service.

I think the content lead is the most natural candidate in terms of control of infra, in terms of collaborator perhaps it should be a curator group? I don't remember if we support that, but if we don't, perhaps we should add it before full synch is ready.

I think our goal is to build tools for people to be able to run their own apps, independently, and not rely on third party to both continue running a service and not be malicious.

I think the DAO lead is not a trusted third party in this capacity any more than everything else they are involved in. I agree that we should make it possible for people to run their own standalone synching, but should we be investing effort in making this technically work well now at launch, given how unlikely it is to be the case that someone right off the bat will invest effort and outlays to operate this at t=0, and also that it may be better if they didn't because lots of things probably need to be ironed out and improved?

I'm also unsure how incentives would work if there was a single YPP backend, who would be paying for those?

DAO should be paying eventually, I think this is more likely to work out well than lots of YPP programs which in parallel go and promise things, which may or may not be honored by the DAO in the end. Having the one backend aligns who is paying with who is doing synching in a way which reduces risk of creators getting bad experiences do to promises that end up not being honored.

[kdembler]

Well, if we are talking about launch, I think we just shouldn't worry about it. I really doubt anyone other than us will be wanting to run a product with YPP enabled, so I just wouldn't focus on this. If somehow someone wants to do this, they still can and we can offer our support. But if we're talking about goals for the future, I think we should design with multiple independent YPPs in mind.

[bedeho]

I think we just shouldn't worry about it. I really doubt anyone other than us will be wanting to run a product with YPP enabled

I'm hoping the gateway WG will indeed run x instances, all with YPP enabled, so I think it is already a question there. I think whether we or the WG lead actually operate it is of secondary importance, I'm good with both, but we should also be using he same single instance in this case.

[bedeho]

Checked, we only support members as collaborators, so created this for mainnet scope: https://github.com/Joystream/joystream/issues/4237

[mochet]

Just to clarify, GWs likely should in the early days be hired, so their rewards are paid by the DAO, this is distinct from rewards to creators signing up through a specific GW, which I think is what your question pertains to.

I think that it does not make economic sense to require GWs to invest in incentivising creator adoption when that generates big spillovers in the DAO which they cannot capture well. The DAO wil able to justify a much greater scale of investment in this. I think that we, as Jsgenesis, are simply doing this in place of the DAO until

Besides the DAO investing in this activity directly, with gateway tokens it should be possible and feasible for GW operators to provide these as an incentive rather than $JOY. They would have an incentive to do this as it helps to build their own userbase, independent of the Joystream platform as a whole. Especially if there are successful gateways and their GW token has some value, or a well regarded userbase this reward as well as any $JOY the DAO may decide to offer, would generate the highest level of incentive.

[mochet]

For example, as a creator tries to use more than one app, each synching services would be unaware that the same channel is already created, or worse yet, they may start replicating videos into the same channel. This does not need to be malicious on the part of any app or creator. A malicious creator could attempt to do so in order to get duplicate rewards, depending on how rigorous the verification is at any given time, which depends on how well verification scales.

This is true. However the DAO could enforce via social agreements that gateways are restricted from onboarding the same channel as another has already done. One way this could be done is generate checksums of each sync'd file, which the Gateway operator then uploads as a text file to their worker static storage. Then the infrastructure/system could be linked to check this storage before allowing a channel to be syncd. If duplicate checksums are found it should give an error.

Another way to tackle this may just be a dedicated gateway that just acts as a piece of infrastructure to ensure there is a trusted list of known sync'd channels. To prevent abuse, there could be one or more of these servers in operation. There are multiple open-source alternatives to airtable and retool that could be effective for this (at some point)

If abuse is found to be happening the GW WG lead should punish noncompliant or abusive GWs.

(above depends on governance and development of tooling, so may not be a real solution for the time being)

[dmtrjsg]

Thanks all for chiming in, based on the comments seems like we are going forward with one for the time being in favour of reduced complexity of getting this off the ground. To be re-evaluated at a later stage.