Atlas product release plan preparation

[kdembler]

Scope for the pre-release plan meeting:

• Identify outstanding problems existing in the app
• Figure out the details of work on responsiveness
• Figure out options for testing Atlas
• (nice to have) Explore options on hosting mock images/videos on Jsgenesis infrastructure

[kdembler]

Outstanding issues

Mocking and business logic

As it is currently, mocking and business logic regarding fetching data in Atlas isn't very sophisticated. Because we won't have access to a real query node in the foreseeable future, we will need to implement mocking alongside the business logic so it can be tested. Current limitations:

• Almost all queries return the same result every time they're run - because the data is not unique, some things like IDs are duplicated across data and we need additional code for example for React key
• Data is lacking some of the attributes - e.g. category that could be filtered
• The dataset is relatively small
• Mocked data is infinite - when one query is run a number of times (e.g. infinite scroll video grid), it will never run out of data. Because of that, some components (infinite scroller for sure) will need additional code to handle cases when we run out of data. An additional issue here is that currently, we don't have a common format for pagination metadata (see Hydra section)

Possible solutions

The first three problems (duplicated and not unique data, possibly even all of the problems if we solve the pagination metadata format issue) could be solved by using some client-side library for generating mock data. This way we could dynamically generate a diverse dataset that would contain all the necessary attributes. We could also do that manually as a one-time shot instead of dynamically

UI polishing

The UI will continuously require reviews and polishing. Some UI issues are already identified (#1082, Joystream/joystream#771), others will surely show up later in development. There are also some things we haven't necessarily discussed yet but probably should be included:

• Error handling - what happens when we fail to load a resource? What kind of feedback does the user receive? How does the UI behave? Maybe that's something we want to ignore for now?
• Loading states - we already have this partially figured out but we're lacking a pattern to handle some cases, i.e. when we are loading but there is no obvious component to display the grayed out placeholder box for. At this moment I think it's only the video playback screen (Loading... placeholder) but there may be other places that will suffer from the same problem
• Make sure all elements are clickable - currently, there are some video previews (infinite scroller, video view "See more") that the user can't navigate through. We probably should add Atlas-specific wrappers to some of the components from the shared package that would contain navigation logic
• Unused elements leaked from the design - we also have some elements implemented that don't really make much sense at the moment I think. This includes: Play and Share buttons on the landing page (what are they supposed to do?) and Share button in video view

Hydra

There is an issue I've identified with Hydra at this moment. The GraphQL framework used by Hydra doesn't yet support returning pagination metadata along with the requested paginated data. This poses a challenge to Atlas - it will sometimes need to know the total number of items to correctly handle the UI state (for example in the infinite scroller). The issue is currently tracked here: https://github.com/goldcaddy77/warthog/issues/338. However I'm not sure what's the time horizon for getting this implemented. The issue also mentions conforming to Relay spec for pagination but we don't currently use Relay-style pagination.

Possible solution

We could switch to Relay pagination format or we could ask the developer to also include support for offset-limit pagination metadata. Either way, we need to agree on the format that we'll be using and implement that for the mocked data for now.

Designer input

There are some places that we may need a designer to figure out how they should look like. At this moment this surely includes:

• Video view - current design assumes an uncommon aspect ratio, see Joystream/joystream#1184
• Alternatives for carousel on smaller screens

Those are the ones that come to my mind at the moment but I think there may be more once we finish the screens that are currently in progress and do a more thorough UI review. Also, some things may come up once we start working on the responsive views so we should take that in mind.

[kdembler]

Responsiveness

So far we haven't really focused too much on the app's responsiveness and that's something we should tackle for the release. We have a fluid sized header on the landing page and I've also started some work on variable-sized video previews but that's about it. Things we will need to work on:

• Proper scaling of video/channel previews. We probably don't need them to be this big on mobile and they should also probably be fluid enough to accommodate different screen widths
• Figuring out what's a good way of presenting videos and channels on the landing page. Carousel doesn't seem right when there's only one item displayed at the time
• Making video player view responsive (mainly the video player itself)
• Adding support for different videos-per-grid values for the infinite scroller (and adjusting it in general)

Keep in mind, more will probably show up once we finish screens currently in progress

Testing

One of the other points mentioned for the release plan was the additional testing we could do in Atlas. At the moment, we do simple mount tests for most-used components, i.e. the testing library will try to mount the component in a simulated browser environment and check if that succeeds. This approach allows catching some obvious bugs, like an error thrown inside the render method but won't be able to find anything more sophisticated, like an error thrown on user interaction of after some loading completes. Therefore, I personally don't find this kind of tests very useful. In general, I'd say it's worth considering how valuable tests are to us at this stage of development of a front-end app. Some alternatives/additional testing we could do:

• More complex tests that take component-specific behavior into account. E.g., simulate clicking a specific button and check whether the component displayed the desired result. I suppose it's possible to test almost any scenario we can conceive but it will require a non-trivial amount of work to write those tests and also add a burden of updating them every time we change the desired behavior
• Visual testing - there are a lot of services/tools for doing visual testing, i.e. matching rendered application against some previous snapshot. This allows catching UI bugs more easily. However, I would advise against it at this stage of development as we will be making a lot of changes
• Logic tests - we could also add tests for some logic used in the app. A good start would be utils we wrote (e.g. formatting number, date, etc.). We probably could also add tests for the mocked server to check whether we get good data from it based on some input. One more thing would be some more complex logic pieces inside components (like fetching data and transforming it in some way)

In general, I don't have that much experience with testing front-end apps so if someone sees other possibilities I'd gladly hear about them.

Hosting mock assets on Jsgenesis infrastructure

We've also discussed hosting media files (videos, thumbnails) for mocked assets on Jsgenesis infrastructure so that they can be easily used in the mocked version of Atlas. I haven't found a perfect solution for this problem yet, but here's what I've been considering:

• Getting some cloud storage bucket (for example AWS S3) and just put all the assets there. Then, get a list of URLs to all these assets and include it in the frontend codebase so that Atlas mocking logic knows what URLs it can use for assets. There are two problems I see with this approach:
  • Asset and access management - we would use a centralized service here so there would need to be a list of people that has to that service and can manage those resources
  • Sync between storage and codebase - we would need to manually keep the URLs in Atlas and actual assets in sync, i.e. make sure any updates are propagated
• Running some API that will serve these assets. In this scenario we could have some endpoint in the API that would give us a list of all the assets, therefore eliminating the sync problem. However, that would require some additional code in the mocking logic and would potentially disrupt the general flow in the app. In this scenario, the asset and access management problem still stands.

In general, I think the best solution would be strictly devops one, that wouldn't require any special kind of access but not sure how to do that at the moment. Personally, I'd lean towards the first option - we're only talking about mocked assets, mostly just for development so we don't need a perfect solution for that and most probably uploading the assets and syncing them will be a one time shot.

[bedeho]

First, off, this was a fantastically well done analysis. It covers a lot of areas, and made me appreciate how many moving parts there are in Atlas already. I have attempted to respond to the points raised, and suggested how we deal, or do not deal, with each point w.r.t. the goals of the next release.

Mocking and business logic

1. "we will need to implement mocking alongside the business logic so it can be tested"

This begs the question of what kind of testing this would be. There are many layers abstractions at which to test, simple unit tests of components and state managment being the most prominent example. The testing which mocking appears to facilitate seems to be manual inspections of how the application looks and feels in various scenarios during development.

Is it relevant to any automated testing?

2. Here I did not actually fully appreciate to what extent these were really problems, as opposed to just not implemented yet because it has not been needed. As for the solution, I think generating mock data can be partially automated, but things for many things it's not clear how, such as realistic image & video assets. I also think that any mocked data should be static, or at least easily deterministically reproducible, so that it facilaites collaboration and review of the same scenario.

I think overall on this point about mocking we should probably

a) Nail down what exact problems we want mocking to do for us in the next release. b) Based on the conclusion in a), define the precise scope of missing functionality that must be introduced in the next release.

UI polishing

General

I think the highest priority here is that we need to identify, as quickly as possible, what UI/UX problems actually require design output that we cannot produce well internally. To this end, I created this issue

https://github.com/Joystream/joystream/issues/1245

It can accumlate all problems we decide are in scope to be solved for the next release, and which require design input.

Specifics

For each category of problem, lets compile an exhaustive list, then triage based on severity and cost of remedy.

Who does this?

I will also do a major review once we have the basic screens done to try to contribute input on what polish is specifically needed.

Hydra

I will try to get in touch with the maintainer to see if this change is realistiaclly going to land before the next release.

Is there no way to sidestep this issue for now by changing how Atlas works short term?

Designer input

Can you add "Alternatives for carousel on smaller screens" to the issue above? Image would be great to show the problem.

Responsiveness

This looks like a good initial list, lets go through them on a call to just confirm that we have the same understanding, and then probably all should be in scope for the next release.

I did not understand this one specifically => "Carousel doesn't seem right when there's only one item displayed at the time".

Testing

I think its very easy to punt on testing when getting started and before going into production, because both the team and code base is small. However, I do worry that we may be setting ourselvs up for a hard time later. I am happy to not do anything serious on this if you both have consensus on this. I am leaving some comments on two test varieties mentioned.

1) Visual testing

This would test that the styling, layout, responsiveness stuff all works. Presumably it could check lots of viewport dimensions, device and browser variations. This is the sort of thing that I think is hard to keep track of whether works well as more features are added and changes are made.

2) Logic tests

Tests pay off most where there is state being managed, and so logic associated with updating state is a prime candidate for testing. My understanding is that this is currently magically outsourced to Apollo Client, so it's not clear to me how complex our own logic is here.

You mention complex logic inside of components: I hope we don't have this? the only stuff I would hope to be in there would be the logic to pick out the right styles and values based on props.

Hosting mock assets on Jsgenesis infrastructure

I think we have to make a decision here based on the conclusion we come up with about our goals for mocking.

[kdembler]

we will need to implement mocking alongside the business logic so it can be tested

This begs the question of what kind of testing this would be. There are many layers abstractions at which to test, simple unit tests of components and state managment being the most prominent example. The testing which mocking appears to facilitate seems to be manual inspections of how the application looks and feels in various scenarios during development.

By testing the business logic I've meant actually being able to write it and see it in action. I don't think at this stage we need any automated testing w.r.t. mocking, but we need to expand mocking so that we can write logic that will be needed to handle all cases that a real query node will put us in.

I agree with your point on the deterministic data set, maybe all we need is just expanding the current data set manually with the help of some faking tool, that shouldn't be too much work.

---

Regarding Hydra and

Is there no way to sidestep this issue for now by changing how Atlas works short term?

We will need this pagination behavior sooner or later. If we want to get a more complete version of Atlas now, we probably shouldn't postpone. Solutions I see: a) do nothing right now, and do that work once Hydra offers the required functionality - this way we don't waste any work, but IMO it's pretty risky to ignore some specific part of how the API works because it may lead to some other incorrect assumptions b) assume some pagination metadata scheme - this way we may waste some work as we may end up using some other scheme in Hydra, but changing the scheme shouldn't be that much work in comparison to introducing new API behaviour

---

You mention complex logic inside of components: I hope we don't have this? the only stuff I would hope to be in there would be the logic to pick out the right styles and values based on props.

At the moment we don't have any complex logic in the components and you're right we shouldn't have any stricte inside the components, more precisely I meant logic used by components to fetch data.

[kdembler]

All tackled as part of Babylon :tada: