Feeds, infinite scrolling, lazy loading and rows with different heights

[oyeanuj]

@bvaughn This library looks really interesting. I was wondering if it can be adapted (or functionality be added) to make it work for the use-case of lazy loading a list of items but without a fixed container height?

My use-case is similar to a feed like Facebook, or Twitter, where I have a list of items to show in the viewport, would like to not render things which are not in the visible area (lets say, defined as the viewport + x) and some support for pagination?

I know that might not be where you see this library headed but wondering if any of the above will be solved by the library.

Thanks again for this!

[bvaughn]

Hi @oyeanuj,

Thank you!

I think you could achieve this functionality with the current release of react-virtualized (unless I'm misunderstanding). Here's how I'd initially approach it...

Firstly, do you know the total number of items your list could contain if the user scrolled all the way to the end? If so, just give react-virtualized that number as the rowsCount. I assume you don't know that number though and you just want to fetch N records at a time and append to a local cache. If that's the case then you could give react-virtualized a rowsCount of X + 1 (where X is the current size of your cache). Then if a user scrolled near the bottom of your list- react-virtualized would request for you to render that (missing) last row and you could (1) render a loading placeholder while you (2) request the next batch of N records. This way you'd just-in-time load records each time the user scrolled near the bottom. Once the new batch came in, just update the rowsCount property and the new rows would get rendered in place.

This may not be what you had in mind though. Maybe you wanted the prefetching to occur way before the user reached the bottom of your list? In which case... you could do something similar but provide an arbitrarily high rowsCount value. Unfortunately I don't think the smooth scrolling would work very well on that. If this is what you're looking for then I would need to give it some more thought.

[oyeanuj]

@bvaughn Thanks for the quick and detailed response! Few more follow-up questions -

1. On the X+1 issue: It would be ideal to have the call to pre-fetch the next set of items a few items before the end. Is it a feature that you think belongs in the library?
2. If that is too hard to add, then I can try doing an 'X+5' rowsCount or in the worst-case, add a load more button which fetches the records. If I go with this approach, two more questions on that -
   • Do I add a custom element as the X+1 element which performs the fetching or are there events in VirtualScroll that I can listen to, that tells me to fetch the next set?
   • Do I update the original list, update the rowsCount and VirtualScroll know to just keep scrolling?
3. Is there a way to use it without specifying the list height? Again, thinking of Facebook/Twitter as an example, the visible list height is really the viewport - some other components? Does it inherit its parent-container's height?
4. Does it do true lazy loading wherein those components later in the list (not visible currently in the viewport) will be added to the DOM/VDOM when its their turn or are they already added? The reason I ask is that I have some actions fired off when the component is loaded, and I woudn't want them to be fired off before (all at the same time)

Thanks again!

[bvaughn]

No problem. :)

1. On the X+1 issue: It would be ideal to have the call to pre-fetch the next set of items a few items before the end. Is it a feature that you think belongs in the library?

Hm. What if I added the ability to register an on-scroll callback? I could call it with a params object that had keys like rowStart and rowStop (indices) and you could choose when to prefetch the next batch of records. That would be relatively straight forward to add. Would this work for your use case?

If that is too hard to add, then I can try doing an 'X+5' rowsCount or in the worst-case, add a load more button which fetches the records.

I think this approach is less ideal because the amount you had to add to X would vary depending on the height of the list and the height of individual rows.

1. Is there a way to use it without specifying the list height? Again, thinking of Facebook/Twitter as an example, the visible list height is really the viewport - some other components? Does it inherit its parent-container's height?

Currently this is not supported. You would need to wrap the list/table in a container that listened for resize events and updated the width/height props. I've considered adding that wrapper component to react-virtualized myself but I haven't wanted to tackle any edge-cases it may introduce.

1. Does it do true lazy loading wherein those components later in the list (not visible currently in the viewport) will be added to the DOM/VDOM when its their turn or are they already added? The reason I ask is that I have some actions fired off when the component is loaded, and I woudn't want them to be fired off before (all at the same time)

Only visible components are rendered. Let's say you have a list that's 100 pixels tall and a row-height of 10 pixels. That means that I will render 11 rows (to account for the fact that the top and bottom row may be partially visible). Then I rotate what those 11 rows display as a user scrolls up or down. (Does this answer your question?)

[oyeanuj]

Thanks again for the quick turnaround and patient answers :)

Hm. What if I added the ability to register an on-scroll callback? I could call it with a params object that had keys like rowStart and rowStop (indices) and you could choose when to prefetch the next batch of records. That would be relatively straight forward to add. Would this work for your use case?

I think so! I could listen for a particular row number based on the row count I previously passed and then make a call when a certain rowStart/rowStop is true.

Currently this is not supported. You would need to wrap the list/table in a container that listened for resize events and updated the width/height props. I've considered adding that wrapper component to react-virtualized myself but I haven't wanted to tackle any edge-cases it may introduce.

This would be really helpful, as I think a lot of people would end up re-implementing this on their own? Another approach (which might or might not be suited to the project) would be to accept a rowCount and use that do the all the calculations (essentially adding up the heights of all the rows? This would make the library work for rows with inconsistent heights (but might diverge from your original purpose?). The other reason why that could be helpful is a case where each row might have a 'Show more' button which expands the row-height? In other words, what is your opinion on rows have same consistent heights?

Only visible components are rendered. Let's say you have a list that's 100 pixels tall and a row-height of 10 pixels. That means that I will render 11 rows (to account for the fact that the top and bottom row may be partially visible). Then I rotate what those 11 rows display as a user scrolls up or down. (Does this answer your question?)

I see. Does that mean you create and destroy the content in those 11 rows or just show and hide?

[bvaughn]

I think so! I could listen for a particular row number based on the row count I previously passed and then make a call when a certain rowStart/rowStop is true.

Okay. I'll add the callback support in a release soon so that you'll be unblocked.

I'll consider the auto-size feature request. I'm not super excited about tackling that one but... if it's something that other people think would be useful as well I'd be willing to do it. Maybe you could request that as a separate issue and we can see if there's any other interest?

I see. Does that mean you create and destroy the content in those 11 rows or just show and hide?

React creates new ReactElements all the time (as representations of the current state of ReactComponents). So I create N+1 ReactElements any time properties change (eg. scroll position).

[bvaughn]

Actually as I sit down to think about this, I realize that I don't need to add any new properties. This is already possible using either the VirtualScroll rowRenderer property or the FlexTable rowGetter property.

Let's say you have a VirtualScroll list with a height of 100 pixels, row height of 10 pixels, and a cache of 100 items. Maybe you want to prefetch the next 20 rows when the user gets to row 80 to make it less likely that they'll notice you prefetching. All you have to do is trigger the prefetch the first time rowRenderer is called with a parameter that's >= 80.

Does this make sense?

[oyeanuj]

Let's say you have a VirtualScroll list with a height of 100 pixels, row height of 10 pixels, and a cache of 100 items. Maybe you want to prefetch the next 20 rows when the user gets to row 80 to make it less likely that they'll notice you prefetching. All you have to do is trigger the prefetch the first time rowRenderer is called with a parameter that's >= 80.

So are you suggesting a rowRenderer function wherein in addition to returning the element, it triggers a pre-fetching call?

I'll consider the auto-size feature request. I'm not super excited about tackling that one but... if it's something that other people think would be useful as well I'd be willing to do it. Maybe you could request that as a separate issue and we can see if there's any other interest?

Will create a separate issue. I figured it might be a slight deviation from the track of this plugin, so totally understand the hesitation.

Thanks!

[bvaughn]

Yes. That's kind of what I'm suggesting. Your rowRenderer function could call an action creator and then it could decide if/when to kick off the prefetch request.

On Tuesday, December 15, 2015, oyeanuj notifications@github.com wrote:

Let's say you have a VirtualScroll list with a height of 100 pixels, row height of 10 pixels, and a cache of 100 items. Maybe you want to prefetch the next 20 rows when the user gets to row 80 to make it less likely that they'll notice you prefetching. All you have to do is trigger the prefetch the first time rowRenderer is called with a parameter that's >= 80.

So are you suggesting a rowRenderer function wherein in addition to returning the element, it triggers a pre-fetching call?

I'll consider the auto-size feature request. I'm not super excited about tackling that one but... if it's something that other people think would be useful as well I'd be willing to do it. Maybe you could request that as a separate issue and we can see if there's any other interest?

Will create a separate issue. I figured it might be a slight deviation from the track of this plugin, so totally understand the hesitation.

Thanks!

— Reply to this email directly or view it on GitHub https://github.com/bvaughn/react-virtualized/issues/25#issuecomment-164950002 .

[oyeanuj]

Cool, that could work. Although part of me feels that having a callback function allows for clearer separation of concerns (rendering vs side-effects) and cleaner rowRenderer function.

Thanks again for the back and forth!

[bvaughn]

Hm. I understand that argument. Let me take another look and unless I see a reason not to, I'll try to push out an update with this functionality sometime today.

[bvaughn]

Added a new property to VirtualScroll and FlexTable called onRowsRendered. This method is called after rows have been rendered and is passed a parameter object { startIndex, stopIndex }.