kdembler
commented
2 years ago Great writeup, useful to reason about the future (not talking about RICE specifically, but just writing each of those down with some reasoning and estimates). I have feedback for some of the points:
- I think the reach of
Transcodingmay be a bit underestimated. You've only mentioned issues with bad encoding which is obviously a big part of it, but this feature would also allow having a single video in multiple qualities. This in turn would be very useful for users on mobile / with slow internet connection. If a creator uploads 4k footage today, it may be unwatchable for some of the consumers. Multiple qualities would allow the client to pick the version that works best for them, so I would say consumer reach isn't negligible here. - I think the effort estimate on
Scrubbing Previewsis too high, if we assume that we already did the work on transcoding, I think this would be closer to 1/2. Remember that community members already did some kind of POC on this, which was working. - On
Embeddability: I think the name may not be fully descriptive here, I would potentially split this into 2 separate epics: a)Embeddabilityas in embedding viaiframeon 3rd party sites. We actually have support for that already. It may not be the final solution but it's usable. For example check video from Atlas playing on https://blog.joystream.org/community-call-2/ One thing we may be missing is adding some kind of UI with instructions to allow setting this up easily, possibly worth doing that short-term to make embedding easier (thinking about thousands of views from embedded videos fromdrog) b)Social Media Previewsas in having previews for Atlas links on Twitter for example. This sprint I've created an epic for this describing some of the problems we will have to solve (https://github.com/Joystream/atlas/issues/1785) and we also created an issue to do a POC of solving this (details here: https://github.com/Joystream/atlas/issues/1780). Depending on the outcome of POC, it may turn out we can have this working without too much effort. We would potentially need some design help before launching the full thing, but that would be trivial task
Also, one general note that comes to mind after analyzing those is that it could be potentially beneficial to start designing content metadata changes, including number of those and do it in a batch even before we start work on any of the epics. Just a wild idea, not sure if a good one though :P
bedeho
Background
We currently have a substantial backlog of things we would like to do for Atlas, as described in the sections
BaselineandWeb3 Specificin the onboarding issue https://github.com/Joystream/atlas/issues/1577. The purpose of this issue is to attempt to apply the RICE prioritisation model to that backlog.Prioritisation: Parametric RICE Score
In this exercise, I consider a range of projects, which probably would be
Epicsin our new process, and attempt to quantify the dimensions of each RICE score. To be specific, I am estimating:Critically, since I have so little information about the utilisation information on Atlas currently, I have elected to parametrise the
Reachscore by valuesXandYwhich denoted the current size ofthat use Atlas over a quarterly period. With this approach, I can separate concerns, and later we a spreadsheet can be constructed where we can explore how values of
XandYimpact ranking.Hence all
Reachestimates, they are of the formkX, orkY, wherekis some literal number. For some projects, I am assuming the project may in fact increase the total number of users, in which casek>1.Finally, a total score is computed by (
ReachImpactConfidence/Effort), but I omit this part of the exercise here, as a spreadsheet seems more suitable, and some points should be discussed further anyway.Consumer accountsDescription
There is no notion of any user accounts on a server, as is conventionally the case with platforms, which would hold private user data. Private data includes things like who you follow, your viewing history, search logs any metadata about your search
Reach
All consumers will be impacted.
Estimate:
XImpact
Impact is nominal by itself, it may help with making usage devices, browsers or clearing of browser cache, more robust.
Estimate:
0.25Confidence
Estimate:
100Effort
Requires introducing a database in
Orionwhich holds information about what users exist, and holds the current client side information which should be relocated, likeOne would also need to add API for reading and writing this. There is an open question of how to do authentication.
Orionas well in the future. We would also need to do some redesigns for this, for example associated with triggering the login/signup when they try to follow someone and they have not authenticated. This approach probably also means that users on mobile could not use Atlas with an account in the browser, most likely, as there is no signer for mobile browsers. It would mean an app would be a must for account features.For estimation I am picking variant 1, for no particular reason. This issue needs to be resolved.
Estimate:
1User playlists & seriesDescription
There is no ability for channel owners to publish playlists, as ordered compositions of videos from any channel, and series, which also have seasons.
Reach
Some substantial minority of
Y, and a large share of the active Joystream specific content creators, do have publishing patterns that simulate playlist like features. I am going to assume all of them would be reached, and that they constitute 10%. Likewise, I am going to assume that we are going to reach a large share, 80%, of the consumer side, because many of them consume Joystream specific content.Estimate:
0.1*Y + 0.8*YImpact
It is useful for organising content in a channel, and also helps discovery quite a bit, as it can automatically key off playlist ordering.
Estimate:
0.5Confidence
Estimate:
80%Effort
Requires no changes to the runtime, does require a new metadata standard for playlists & series, as well as query node support to decode this standard, which will be quite trivial, but then the work on the Atlas side will be more substantial in terms of design and tech. The impact will both be on the consumer and the studio side.
Estimate:
3TranscodingDescription
There is no ability for uploaded videos to get converted into a range of different resolutions and encoding formats, making it easier for users to consume the content in a way which is suitable for their device and product.
Dependency
Appears to depend in some way on
Consumer Accounts, possibly.Reach
Anyone publishing will eventually run into this issue of encoding related problems, unless they are re-uploading/synching their content from YT, where is has already been processed to work well for browser based playback. On the consumer side, people occasionally will run into videos that have bad encoding, and that would stop, but this is a much negligible effect.
Estimate:
0.8*YImpact
Estimate:
0.5Confidence
Estimate:
70%Effort
Would require
Orion, which is unclear, as seen in discussion above.Orionto accept, store, dispatch work to some cloud transcoding service, like AWS Transcoding, monitor progress, expose this progress in an API for end-user, and complete final upload to storage system when done.Orionon these uploads.Estimate:
5Scrubbing PreviewsDescription
There is no ability to generate image extracts from regular intervals in a video so as to provide content previews while scrubbing.
Dependency
Can only be done after
Transcoding.Reach
All consumers would use this.
Estimate:
XImpact
Estimate:
0.25Confidence
Estimate:
60%Effort
Once we have done the
Transcodingwork, the extra work here isOrionto accept, store, dispatch work to some cloud transcoding service for image extraction, and complete final upload to storage system when done.Estimate:
3RecommendationsDescription
There is no way to recommend to the user what they should watch which is based on either their own or other consumption statistics. This is required in order to for example suggest new a new video when playback of one ends, or to show related videos.
Dependency
In principle this can be done, at least as an MVP, without
Consumer accounts, just havingOrionuse global traffic statics about what recommendations convert, however anything sophisticated likely willConsumer accounts, for this reason I am just assuming the most naive simple version withoutConsumer accounts.Reach
All consumers are impacted
Estimate:
XImpact
Currently it has a limited impact because catalogue is so small, but once we get YT-synch going it will be much more impactful. Since I assume that happens in Q1 2021, I am accounting for this.
Estimate:
0.5Confidence
Estimate:
50Effort
Will require new state in
Oriondatabase representing information needed to power recommendations, as well as whatever logic and APIs that expose recommendations and update them. I am hopeful that the naive version of this can be done with our current team, but at some early point after that, someone with a specialisation in this needs to be brought in to at least advice us on a more useful setup to support further innovation down the road. Changes in Atlas will be negliible design wise, and only modest changes are needed to actually read these new recommendations, as well as to provide extra information toOrionthat updates recommendations.Estimate:
2Query Speed & Quality: Transition Hydra -> SubsquidDescription
Atlas is a super slow application to load, and to my understanding, this is in substantial amount caused by the query node being very slow, despite the fact that it has a very light load. This is because
Hydraas a framework is very under-optimised and under-maintained. Subsquid is a version of Hydra which is being actively maintained by a separate team, and they have already made substantial progress on improving it's query, and processing, speed. There is also the issue of the quality of free-text search results, currently, they work very poorly. In the long term, I don't actually think search across the content catalogue will be powered by the query node, it will likely be powered by some more sophisticatedOrionservice, but some types of search possibly would, such as within a channel, or searching across membership handles.Note: this was not described on onboarding document. Note 2: It should probably be verified that a big part of Atlas slowness is actually due to this.
Reach
All consumers and channel operators.
Estimate:
X + YImpact
This has a very big impact, the slowness is noticeable on the very first interaction with Pioneer.
Estimate:
2Confidence
Estimate:
95%Effort
The primary effort is outside of the Atlas team, however, the Atlas team will need to do quite a bit of testing and collaboration trying to resolve problems with the Subsquid team, possibly finding workarounds if something is not feasible with Subsquid even in the long term.
Estimate:
0.5Better searchDescription
There is no way to search based on anything except a very bad full text matching of the search string against, video titles and descriptions. Sophisticated search will be sensitive to many more parameters, including popularity and consumption statistics of candidate matches.
Dependency
Same as
Recommendations, at least as best I can gauge it.Reach
A substantial minority share of the consumer side use search for discover
Estimate:
0.2*XImpact
Estimate:
2Confidence
Estimate:
80Effort
Very similar to
Recommendations, it's not clear how substantial the marginal work for the naive version of search is, givenRecommendationsbeing done, or vice versa.Estimate:
2SubtitlingDescription
There is no way to display subtitles in content, or allow control of. As a result of the lack of post-processing, there is also no way to have subtitles automatically extracted.
Reach
A large share of the Joystream content at the moment would benefit from subtitling, and community members are currently doing this my manually adding it into the video (example https://play.joystream.org/video/8266), which is labour intensive, and also only allows you to do it for one language.
Estimate:
XImpact
Estimate:
0.5Confidence
Estimate:
70%Effort
There are really two parts here, one is automatic subtitling, and the second is just being able to have subtitling metadata associated to content, and have it displayed on playback. I will only concern myself with the latter, as the former requires a lot more work, and it depends on the latter anyway. On the design side, we would need to introduce some new part of the upload/editing flow for content for associating subtitle metadata, and we also need to adopt some off-the-shelf standard for this. I did some basic review and
SRTseems to be the most popular format, and it also is compatible with YT, so when we do YT-synch, perhaps we get that format coming out already:https://support.google.com/youtube/answer/2734698#zippy=%2Cbasic-file-formats
Payload needs to be stored as separate assets in storage system. The playback side has already been designed. On the tech side, changes would be limited to accommodate this.
Estimate:
xComments & LikesDescription
Threaded discussion, per video, with reactions on the video and each comment, moderated by channel owner.
Reach
Nearly all consumers we currently have would be impacted by this, both using and seeing the feature.
Estimate:
0.8*XImpact
I think this would have a big impact currently, in particular given in context credit and community feedback on content, I have personally wanted to do this very often. It also would allow for discussion and async asking questions and support.
Estimate:
1Confidence
Estimate:
80Effort
The runtime side of this has already been completed, which is where this state lives. There is no query node support, but it would be extremely easy to add, 1-3 days really. There is no design done here, which would be non-trivial.
Estimate:
2EmbeddabilityDescription
There is no way for users to embed playback of a video on third party websites or social media.
Reach
A lot of our active creators would want to share their content more effectively, in particular in the future if there are rewards tied to consumption data.
Estimate:
0.1*YImpact
Estimate:
1Confidence
Estimate:
60Effort
I have no clue!
Estimate:
?Creator Social TokensDescription
The ability of creators to monetise their channel by sharing their future channel revenue with their most passionate fans, as well as give them access to unique content. Here are some very rough, and now dated, requirements
https://github.com/Joystream/joystream/issues/2362
Reach
Likely would radically increase interest on both the viewer and creator side, only damper is that it would not be truly for real yet, as we are not on mainnet.
Estimate:
X*5 + Y*10Impact
Estimate:
2Confidence
Estimate:
90%Effort
Here the runtime work has not started yet, and that will take some time, and there is no query node support. Design would impact both the consumer and viewer side, in a very similar way to the NFT work as done. It may be possible to try to make earlier progress on this work by trying to work top-down, and nail down requirements on the product side before the runtime is done, or have it happen in parallel, and that would have the added benefit of allowing more market research and product input on how the system should work.
Estimate:
10Reflection
In the end, I am actually not sure the ranking would look very different if I did this informally without RICE, in particular because of the dependencies imposing extra constraints, and I am also a bit concerned about the reliability of a lot of my impact and reach estimates. Put it differently, if the output of the analysis results in the conclusion that we don't do comments & likes first, then I would question the numbers I have estimated :D. Perhaps the framework would be more useful when considering a very large number small alternative changes that have limited internal dependencies.
One thing I did find useful was the exercise of identifying, or attempting, to the total amount of effort required across all the things we are considering doing, which is always humbling, and I also suspect the output of these notes will be helpful for the team to get more informed on where we are going.
Next Steps
I think the team should review everything, just to absorb the information, and then we need some validation from the tech team about some of these estimates, before we have final credible RICE scores.