Scrollview with different layouts + infinite scroll

[kof]

I missed any example of how to create scroll view item which consist of different surfaces and layouts.

Is it possible? I had no luck to make it done.

It would be also important in #73 for dom reuse, because passing content with a bunch of html into infinite scroll is not good for performance and memory.

[michaelobriena]

Yea, the sequenceFrom function of the scrollview can take in an array of any type of renderables. Your array can be Surfaces, ImageSurfaces, Views (that have a size), RenderNodes, ContainerSurfaces, etc.

[kof]

So if I put a ContainerSurface which contains a bunch of other surfaces into sequenceFrom array, all dom nodes will be reused. If I put just one surface with content string which contains all the other elements - all the inner elements will be recreated for every item, right?

[michaelobriena]

Kind of, but you don't need a ContainerSurface for this. If you were to have a Scrollview with a ContainerSurface that has many Surfaces inside of it, we can't reuse those Surfaces until the entire ContainerSurface is outside the Scrollviews margin. Just have the Scrollview sequence over the collection of Surfaces.

The margin property of the Scrollview is what determines when the DOM elements assigned to the various Surfaces are freed up for reuse. When a renderable is outside of the margin, we free up that DOM element and it is reused.

When it comes back into the viewport, we see if we have any free DOM nodes to use, if we do we use it, if not we make a new one. We then set the innerHTML (or appendChild) to that DOM node with the Surfaces associated content.

All in all, we get to reuse the DOM nodes the different types of Surfaces manages but can't reuse the inner content.

Sorry if that was overly confusing.

[kof]

It is a bit confusing, let me sum up this:

It doesn't matter if I pass ContainerSurface with surfaces inside or Surface with html inside. The only element which gets reused is that one passed to sequenceFrom, all the inner elements will be recreated every time the sequenced surface is rendered. Correct?

[michaelobriena]

Not every time it is rendered but every time it goes from not being rendered to being rendered. But yes, only the DOM elements the Surface (of any type) manages is part of the reusable pool, not the content.

[kof]

Ok lets say on deploy :) So can I create view for every list item, add all the surfaces into the view and sequenceFrom views list and make it reuse all the dom nodes?

[kof]

Do I need to use Group to get the scroll transform applied on all list item surfaces?

[michaelobriena]

Yup, correct.

The only thing worth noting is that having too many Surfaces is bad since we have to reset the matrix3D property on all Surfaces and overuse can hurt performance. Anything that does not animate independently should usually be a single Surface. The performance loss of setting the innerHTML with a large amount of content at deploy time is much much less than overusing Surfaces.

To your second question, try taking a look at the "setOutputFunction" for Scrollview that essentially changes how the Scrollview lays out it's collection of renderables.

[kof]

All in all it sounds like its not ready.

1. One surface and content with html string in case of f.e. 10 dom elements means on scroll it will be dozens of dom nodes created via innerHTML which is not really fast
2. ContainerSurface - same like previous + javascript overhead for creating all the inner surfaces
3. View with Surfaces - we get a node reuse but at cost of setting matrix3D for every surface, which is higher than in case 1

It really sounds like its broken by design.

[kof]

It should really be possible to have the case 1 or 2 with nodes reuse. I got it even in iscroll.

[kof]

Not fixing this means the whole strategy for dom reuse is a shit, because 1 list will cause more created dom nodes than the whole application.

[andrewdeandrade]

@kof What's up? Just chiming in here to give you a better mental map of how everything works together, so you can figure out what each component does and why things were designed the way they were. I'm going to start at a low level and work my way up.

First, famous is designed around the notion of retain-mode graphics and a scene graph (which we currently call a render tree). Basically, retain-mode (versus immediate mode) is a way of handling scene graph modifications in memory and then deploying them to the screen on a fixed interval. DOM is natively immediate mode. When I set a property, that change happens instantly over the next few clock cycles. This is bad for the perception of visual performance because there is no intelligent control over what happens and when it happens. The burden is 100% on the developer to program things to the best of their capacity. Retain-mode affords developers the ability to declaratively state what transformations they'd like (which changes the model in memory) and then the famous "engine" (specifically engine, specparser and other primitives) are responsible for deploying those changes to the DOM. These low level components make sure that the minimum number of DOM mutations are necessary and that these DOM mutations are performed in the order that is fastest and causes the fewest performance harming side effects.

With those low level components described, let's move on to some of the higher level components addressed above. First let's start with the idea of a scroller. In famous, the core scroller component is really very primitive and intentionally so. The core scroller effectively acts as a controller that accepts a linked list of renderable objects (i.e. a mathematical set), maintains a cursor to the current position in that set (the first item by default until the internal state changes). Furthermore, the core scroller supports the notion of a "master output function" which is basically a function that accepts a renderable (with an internal state in the form of a transform and size) as the input and them mutates the state of that renderable based on the position of that renderable relative to the current cursor position).

With this in mind, you can use a SequenceView to build up a linked list of renderables (anything from simply surfaces all the way up to a complex view with many surfaces), that can be of the same view/widget type (like a list of tweets) or all different (like all the different types of stories in a facebook feed).

Now let's briefly address limitations relative to the quantity of surfaces that Famous can performantly manipulate in the current crop of modern browsers. On a decent desktop or laptop, I've seen some stress tests get up to 400 surfaces being manipulated simultaneously and all being in the viewport. This is more than enough for all but the most demanding cases. Why? Because the likelihood that you have (1) 400 items in your scene graph, and (2) in your viewport, and (3) all moving at once is really really slim. Even 50 to 100 is unlikely except on large screens.

How does all this relate to the scrollview? I can create a scroller to display an infinite list, I would use a controller that feeds them into a controller that exposes a ViewSequence-like API (i.e. "link" them all together and expose an API with get next and prev that the scroller knows how to move through) and then give that sequencing object to a scroller and display that scroller in a context and have it work performantly. So long as I don't set up the scroller properties relative to the viewport in a way that more than 200-400 surfaces are visible at a time it should work. If I only have 10 surfaces visible at a time relative to the viewport, then the engine really only does the work of manipulating a little more than 10 surfaces.

Now, this rule applies to simple surfaces, where those surfaces are basically static (they can contain complex DOM, but that DOM is basically unchanging). Now you ask "What if I want something more complex than a static surface?". Well then you have a multiplicative effect. If you have 10 views visible in a view and each of those have 3 dynamic parts, then you are deal with ~30 dynamic components in the viewport. If each has 20 items, things get hairy since we get back up to the 200 simultaneously manipulated surfaces, which starts to become problematic.

So, given these concepts, how should one go about working with the scroller with complex objects and not lose their mind? First, always think about things in famous in terms of objects/elements that can be independently changed (i.e. affine transform and opacity). If I have two DOM elements that buzz around the screen each according to their own movement function, then those should be allocated as two views. If however, I have a post it note with a title and body copy and those always move together and are shown or hidden together, they should go together on the same surface. By putting them on the same surface, you make the Famous engine only perform the work (calculations and manipulations) on one node in the scene graph.

If I got things right, the issue you're mainly concerned with is the work of deploying and redeploying view objects in your scroller. The current interface for a surface is very naive and intentionally so. It let's you pass in a string representation of the DOM you want. When scrolling fast enough on view sequence items that have a reasonably complex HTMLDocumentFragment to be rendered, you're likely to encounter some performance issues. This isn't an issue or limitation with the scroller or the view sequencer or surface itself. Instead this is suggestive that there is yet another place where there is room for additional optimizations that what we've yet had the opportunity to build into the framework. How are people likely to solve this problem now?

(a) they could try to solve it with famous, but continuously partitioning their views into more and more surfaces. This naive approach is not desirable because it's bad for performance (more visible nodes in the scene graph being manipulated) and because its unnecessary. If these further partitions result in visible elements that move together (i.e. same affine transform operations) or "opacitate" together (i.e. all have the same opacity at any point in time), there is no value to putting them in the scene graph.

(b) The other naive alternative is simply regenerating an HTMLDocumentFragment string using a well known client-side templating solution like handlebars or ejs. This will get the job done, but is less than desirable since it causes the browser to do a lot of work, since it needs to take a string, parse it into an XML representation in memory, used that XML object to construct the document fragment and all its children and parse the properties of each XML node to set IDs and classes, cascade CSS properties to style it, etc. This is an awful lot of work to do every time you deploy a surface.

The smart approach, which we're investigating is creating a templating solution that can be modified at the DOM node and property level. i.e. we construct the DOM fragment once and any changes to the model that backs that object propagates to only changing the minimum nodes on the DOM necessary as opposed to fully recreating that entire fragment for the sake of changing one or two of the leaf nodes. We do not yet have our own solution for this, but some solutions out there may already work out of the box. Two that I am aware of (but have yet to try out myself) are Meteor's Blaze templating engine and Ember.js' HTMLBars templating engine. IIRC, both of these attempt to address the performance implications of recreating larger chunks of the DOM for the sake of one or two elements. Furthermore, I wouldn't be surprised if React.js were also usable in this intra-surface context for diffing DOM efficiently. As someone who has maintained a templating engine for HTML yourself, I'm sure you'll appreciate the nuance here between what Famous does well and HTML templating system with atomic updates that will complement us well.

All in all this is not broken by design, and the design is very much intentional. This is something we've been aware of for a long time and we've been keeping strongly to small composable parts that have single responsibilities. The issues you point out are responsibilities that need to be handled, but they should be handled by specific components that handle those problems well instead of overloading components like Surface with that functionality.

We're not avoiding a fix here because these components aren't broken. Famous is beta and already mature for many use cases. For some use cases, like yours, we're working on our own additional components that solve your use case, but if your impatient, you can take the components we have and use them for what they do well and couple them with other solutions out there that address the problems we have yet to address. The only difference between those solutions and what we're going to build largely will come down to API consistency and ease of use. Under the hood, the solutions are going to have a lot in common. API-wise, once we have our own primitives for solving these problems, we'll make it so you don't have to actively spend your brain juice on choosing when to use a surface or a view or a templating engine with the ability to perform atomic DOM mutations.

Anyways, I hope this helps clarify things. Let me know if you have any further questions. If you do decide to explore Blaze, HTMLBars, React or your own solution for intra-surface atomic updates, please report back. We'd love to hear what you discover.

[dcsan]

nice explanation. one small point

if your impatient, you can take the components we have and use them for what they do well and couple them with other solutions out there that

famous makes it pretty hard to mix and match other components. issues like breaking the native scroll seem like the design is to be all or nothing:

https://github.com/Famous/core/issues/14

perhaps when you have your own wrapper some of those decisions may become clearer?

[andrewdeandrade]

@dcsan Totally understand your concerns with "all or nothing". There is some truth to that statement, but I think that overtime, this will become less of an issue over time, and for the specific case outlined above is not exactly the case.

We (royal "we" as in all web devs) have been abusing the DOM for years. First, we used tables to achieve complex layouts, then there was the backlash against tables that pushed us to use DIVs instead of tables because they are semantically less meaningful and thus a better blank slate object to work from. Whether it was table hell or div hell, we've been producing frameworks for the better half of the web's 25 year existence whose job is simplifying the management of this hell. Given the shear quantity of these solutions and the number of libraries that bundle div hell management as a core piece, it's understandable that our radical departure from these approaches lacks compatibility with many libraries out there. On the other hand, now that this solution is here and here to stay, you'll find that although famous doesn't play nicely with libraries from the past, it will play nicely with libraries being created today and that will be made in the future.

Case in point: React.js. ReactJS shares a lot of things in common with Famo.us. In fact, we've been talking to the core react people for a long time now. @petehunt (a core author/committer) paid a visit to our offices and was able to cook up a naïve integration between Famo.us and React together with @dmvaldman. They can correct me if I'm wrong, but I think it took them a 1-3 hours and 100-200 lines of code to do most of the integration. Over time there will be more and more libraries that adopt some of the same techniques as us and for the most part they should play nicely with a little work.

FWIW, we're now actively devoting time and effort to integrations with other popular modern javascript libraries and frameworks out there (react, angular, ember, d3, backbone, etc). We're also looking into how to reduce the "all or nothing" perception and reality. Let us now how we can improve this. If there is something specific, we're all ears. If you've got a proof of concept, even better. So long as these solutions don't comprimise some long term goals we have, we're open to making famous more compatible with what is out there.

Lastly, with regards to the issue in this thread specifically, Handlebars and EJS work just fine. I don't see any technical reason off the top of my head as to why something like Blaze, HTMLBars or ReactJS could not be leveraged to solve the problem described above so long as the mutations being made are not affine matrix or opacity mutations.

[kof]

@malandrew thank you for this explanation.

I think the biggest problem at famous is the lack of documentation of low lever stuff like this.

I know exactly what you mean by DOM diffing, I have started an experimental lib for this https://github.com/kof/diff-renderer it is not ready because of lack of time, but I have a concept how to make dom diffing independent of template engines and being able to change attributes, remove nodes and resort nodes fully automatically and out of the box.

DOM diffing would be a more developer friendly solution, one needs just to pass the new html string. But it also has its cost.

A way more effective for this high performance use case would be to update all the inner nodes of a surface manually when it is in the viewport or within margins.

What I miss is:

1. How to hook in when a surface is in viewport or within margins.
2. How do I know which position in the global sequencedFrom list it has. Knowing this I can maintain my own nodes pull update them and inject them into the surface manually.

[kof]

In short: we need an event which fires every time when a list item gets rendered and passes its renderable instance. So I can grab the item id and set the content using a documentFragment.

[kof]

Wait, I believe I know how I can do it right now:

'recall' event is fired when a surface is cleaned up. In that moment I can release my documentFragment and set state occupied = false so it can be used on the next surface with new data inside. I build an own fragment/elements allocator which will give me a new fragment with my elements inside if there is no free one.

Please correct me if I am wrong.

[kof]

I can actually perfectly make use of ElementAllocator class for this.

[andrewdeandrade]

@kof No problem.

We're working on documentation. If you feel that there is something specific where you think the documentation is lacking, please by all means open up a git issue asking for someone on the team to dive in and clarify things. We're here to multiply the effectiveness of each other and members of the community like yourself. If you're not effective because docs are missing, let us know. We'll address that.

What would be most useful would be to go through the source and put comments in. One approach is to fork the repo in question and then, in your personal fork, comment directly on the source code using "@" mention with the handle someone on the team to get them in on that discussion.

Regarding the DOM diffing approach, we'd actually like to take things one step further. First, we want to get developers away from modifying HTML at all for anything dynamic. HTML content is always either static or always "instantiated" once with some JavaScript interface for updating the content. Until the browsers provides a perfomant alternative (which may not be possible due to path dependency), touching the DOM directly is usually death to performance. Instead we should let JavaScript handle the applications of all mutations for us. It is in that interop layer that we can program everything to be efficient and fast. The way we see it, HTML is valuable to help people structure semantic content (per the original intent of hypertext) and that from that HTML we can generate JavaScript objects that give us the interface we need. Better than strings of HTML is just using POJOs (plain old javascript objects) that either describe the values to be updated in the DOM or function as references to other POJOs and the corresponding template objects. These DOM managers can be recursively defined and the developer need not worry about implementation details of how DOM nodes are mutated or how DOM children are attached, removed or otherwise manipulated.

I am by no means the lead architect on the famous framework, so don't be surprised if @dmvaldman @marklu @ftripier @michaelobriena, etc, correct me above. Hopefully they will add more to the discussion here, either addressing your need or suggesting an alternative.

[kof]

@malandrew yeah, you will probably end up in Surfaces which can be diffed to their SurfaceContainer. What I have in my diff-renderer is additionally a hyper fast html parser (not the robust and forgiving one) which allows compatibility to all templating solutions. After html is parsed and a json tree is generated its exactly the same what you would implement. Diffing 2 json tries and applying changes.

[petehunt]

What @malandrew describes is exactly what React is: a way to take an immediate-mode view hierarchy of components composed together and render them to a retained or immediate backend. If you want to go with this approach it's probably best to start with famous-react (way incomplete but a good proof of concept) since we already figured all of the main edge cases with this approach and built the heuristics needed to performant.

[kof]

@petehunt have you tried to compare applying diff on a documentFragment with lets say 10 elements (changing textContent and attributes only) using react vs. pure dom. How big is the overhead?

This would answer the question if using react within high performance infinite scroll on mobile devices is a good idea. My feeling says there will be some milliseconds too much ...

[petehunt]

This is an enormous topic, but the approach naturally lends itself to windowing (only rendering what's in the viewport) and DOM node recycling. This means that there's overhead relative to manual management, but it buys you a lot. And it's much much more performant from a memory and often CPU perspective than an observable approach.

A good discussion is here: http://calendar.perfplanet.com/2013/diff/

[kof]

@petehunt We are talking in context of famous Scrollview, which renders only stuff within viewport + margin independent of the rendering solution. The only thing which counts now is the overhead of applying changes.

[kof]

I saw this react explanation already, its all fine for react in general. Here we have a very specific high performance case.

[petehunt]

You just pass it a bunch of Surfaces and it takes care of the windowing for you, right?

[kof]

yep.

I would love to see a pure bench of pure dom vs react on same number of elements with same changes. Probably without resort and node creation, just textContent and attributes.

[petehunt]

I think I need to understand the API of the scroller. Do you give it surfaces or a callback that lets the scroller ask for your surfaces?

[kof]

you give it renderables (surfaces, views etc.) ...

[kof]

Here is a working example for infinite list on top of Scrollview https://github.com/JonnyBGod/famous-infinitescroll

[petehunt]

@kof I don't understand how that can be truly infinite if you need to pass a viewSequence that isn't lazy

[kof]

@petehunt yeah didn't got it too first time. You can push to the sequenceArray after Scroller is created.

[kof]

about truly infinite there are still some open questions https://github.com/Famous/famous/issues/73 :)

[petehunt]

@kof so your sequence array would grow infinitely unless you windowed it, which is what the scroller is already doing for DOM nodes. Seems redundant :)

I would imagine emulating the UITableView API would make sense here, since that is a retained-mode infinite scroller.

[kof]

@petehunt its not mine :) ... this is what I don't understand too and asked in #73 I believe famous will release a truly infinite scroller with all that points tackled later.

[kof]

Regarding bench for the overhead rendering diff within a surface with react. I just found one bench which compares react diff vs. innerHTML. innerHTML parses xml creates all the nodes etc etc. it should be WAY slower than maintaining nodes pool manually, but still its 70% faster then reacts diff http://jsperf.com/react-vs-pure-js/9

[dmvaldman]

To clarify viewSequence is a combination of a linked list and stream data structure. It's lazy in the sense of a stream with tail and head where what tail is at any given time is ambiguous. In viewSequence we have previous and next, but the backing array is ambiguous, which allows for lazy loading.

@kof you are correct that it is a current weakness in our platform that we recreate inner DOM nodes on deploy, and that these are not "stored" somewhere (either in a document fragment, or in the DOM itself). As you and @petehunt are well aware, this is a tricky problem. I'd love to hear ideas for solving it. I think React especially has great ideas here.

[bunnyhero]

@malandrew: thank you for the informative comment! it would be nice if this information were in the documentation somewhere, maybe in the guides, or in a wiki or something.

i'm brand new to famous and looking at the reference docs as is, i'm still not sure exactly how Scroller, ScrollView, and ScrollContainer relate, or how to best use or customize them.

thanks!

[markmarijnissen]

@dmvaldman: wait, isn't the content stored now in a DocumentFragment?

[MylesBorins]

@wgester do you want to take the lead on this thread?

[michaelobriena]

Closing as this is stale.

[davidpfahler]

@michaelobriena Is there a solution to this problem, yet? This seems to be THE barrier for web apps at the moment.