Infinite scroll in reverse order

kof commented 8 years ago

Hi, first about my use case: I am building a chat app. Which means it is essentially important to:

be able to render a huge amount of rows
be able to request additional rows
be able to jump in the middle of a 1 year long conversation

This means that the list we are going to render is potentially just a fragment of a very long list, there are rows after what we have got and there are rows before.

We need to be able to scroll up and load previous rows as we go, as well as to scroll down and load next rows.

kof commented 8 years ago

So the missing part is right now the ability to scroll up (beginning of a vertical list) and request previous rows.

To go more technically:

when adding rows at the beginning of the list, scroller stays at the same place, it needs to go scrollTop + addedRowsHeight
InfiniteLoader logic needs to scan for unloaded ranges correctly considering scroll direction.

kof commented 8 years ago

This means that startIndex in InfiniteLoader can actually become a negative number.

bvaughn commented 8 years ago

Hey @kof,

Reading this feature request, I wonder if you've considered doing something like...

Listen to onRowsRendered to detect when row 0 is displayed. (This would be your tip to load older rows.) Row 0 might be an actual chat log message or a placeholder "Load older messages" indicator.
When a set of older rows finishes loading, add them to the beginning of your List/Array- and give VirtualScroll the updated list count along with a scrollToIndex that points to the previous row 0 (eg. scrollToIndex should be the length of the newly-loaded rows).

I think that would work as you've described?

If not, I think it may be appropriate for you to create a fork of InfiniteLoader for your specific use case. I don't think it makes sense for the HOC to support negative indexes in the common case.

kof commented 8 years ago

I have created my IniniteLoader based on the original one. However I can't imagine this is an edge case. This is nothing else but a very long list where you jump in the middle of it and want to go back.

Imagine google search with 1000000 results and you can't load them on. Now you are on a page x and want to go back. With pagination you just click on the previous page number. Exactly the same is here.

kof commented 8 years ago

When a set of older rows finishes loading, add them to the beginning of your List/Array- and give VirtualScroll the updated list count along with a scrollToIndex that points to the previous row 0 (eg. scrollToIndex should be the length of the newly-loaded rows)

This was my first idea and resulted in junked scrolling, because:

I start loading items before the first row is rendered, same logic we have right now, uses threshold param.
Once loading is done, scroll position has evtl. changed, so now I need to set scrollTop = newRenderedRowsHeight + previousScrollTop

bvaughn commented 8 years ago

Imagine google search with 1000000 results and you can't load them on. Now you are on a page x and want to go back. With pagination you just click on the previous page number. Exactly the same is here.

It doesn't seem like the same thing to me @kof. Google doesn't show you the middle results as page 0 (or 1).

Everything component and utility in react-virtualized is written to work with a list or array- things that don't make sense when we start talking about negative indexes. Despite what you say, yours is the first request I've ever received to support negative indexes.

kof commented 8 years ago

Google doesn't show you the middle results as page 0 (or 1).

Well imagine you got a link to the page 100. You don't load all the results before, right?

Everything component and utility in react-virtualized is written to work with a list or array- things that don't make sense when we start talking about negative indexes

Well maybe negative indexes isn't the right approach anyways. My interest is to unlock the use case. How is a different question.

bvaughn commented 8 years ago

I mentioned a potential workaround using scrollToIndex. I understand your concern about scrollTop maybe having changed while the older chat records were loading but I still think that's the best approach. Maybe an improvement to my original suggestion would be...

Treat row 0 in your list as a place holder and render a "Loading more rows..." in it.
Listen to onScroll to detect when scrollTop is 0. When this happens, fire off a request for the next chunk of older rows.
Keep listening to scrollTop while your fetch is in progress.
When fetch completes, set scrollTop equal to the height of the newly-loaded rows plus the value of scrollTop from the most recent scrollTop call. (This should account for the case of the user scrolling back down to view newer messages.)

There are some limitations of how react-virtualized can be used. That's because I've written 99% of the code and I only have so much time.

Feel free to propose (and submit) a new HOC if you think there's an alternative to InfiniteLoader that might be able to better handle this.

kof commented 8 years ago

understand your concern about scrollTop maybe having changed

additionally threshold option would mean that loading is started before scrollTop is 0, once everything is loaded I would need to know how many rows will be rendered to scroll to the last known position.

kof commented 8 years ago

set scrollTop equal to the height of the newly-loaded rows

not exactly right, we might render less than we load, which means I need to calculate height of newly-rendered rows.

bvaughn commented 8 years ago

additionally threshold option would mean that loading is started before scrollTop is 0, once everything is loaded I would need to know how many rows will be rendered to scroll to the last known position.

So implement your own threshold ? (Rather than checking for scrollOffset 0, check for 100- or whatever else you want to use for a threshold.) I don't think that's too much burden to put on application code.

not exactly right, we might render less than we load, which means I need to calculate height of newly-rendered rows.

That's not how RV components work. To render (and position) row N you need to have measured rows 0...N.

bvaughn commented 8 years ago

I don't think there's any action for me to take on this issue at the moment so I'm going to close it. We can continue discussion though- I just like to keep the issues list pruned.

jamesbillinger commented 8 years ago

@kof I'm needing to do the same kind of thing. Chat...which should really be bottom to top scrolling. I'm curious if you ever found a workable solution...I managed to get variable height rows working by pre-rendering in componentWillReceiveProps (I couldn't get CellMeasurer to work with redux), but I'm stuck on the bottom-to-top layout.

It works fine if I just download all messages in a conversation up front. I'm not sure at what point this will start to be a performance concern. It would be ideal if I could structure my array newest-to-oldest and just have the RV component render bottom to top. No idea what that would take. Anyway, I was just curious if you had suggestions.

kof commented 8 years ago

I have one, but I still didn't come to make it ready for open source.

kof commented 8 years ago

Also my requirement is to load infinite amount of messages and also being able to start in the middle of that stream. I have implemented a bunch of components to do that.

jamesbillinger commented 8 years ago

No worries...and thanks for the response. I think for now I may just load the last 100 (or so) messages, and then offer a button to load all (slowly) if need be.

smhg commented 8 years ago

Another example of the same bi-directional scrolling need: calendars. react-infinite-calendar solves this by having far-away start- and end-dates. Not that big of an issue for a calendar, but still a hack. Could a new HOC address this? I'd be happy to give it a try. Or are there too many insurmountable issues?

bvaughn commented 8 years ago

The biggest issue I can see- other than the added complexity from additional forking behavior- would be with dynamically measured content (eg a chat list using CellMeasurer).

react-virtualized has been optimized to defer measuring cells until needed for better performance. If you have a list of 1,000,000,000 rows, but only the first 15 are visible, it only measures the first 15 and uses an estimated height for the rest. If you scroll down to rows 50...64, it will measure 15...64 but then continue using the estimated height for 65+.

The reason it measures rows 15...64 if a user jumps from 0...14 to 50...64 is a bit complicated to explain. Basically once react-virtualized measures a certain cell- its size and position is cached (unless certain properties change which tell it to clear the cache) to avoid having to recalculate it later. (This is another performance tweak.) If we were to continue using estimated row heights for rows 15...64, then as a user scrolled up from row 50- we would have to run through every row after the current row and slightly adjust their position to match their newly-calculated position. (This gets very expensive if a user jumps to the end of a list and then scrolls to the beginning.)

Now, some caveats:

We could skip this update if the estimated and actual row height were the same. (Meaning fixed-height rows could avoid this performance cost.)
We could try alternately just throwing away pre-computed/cached rows after an updated one (eg if you jumped from 0...14 to 50...64 and then 30...44- we could throw away cached positions for 50...64 and just re-calculate them later if needed).

I have thought a little about doing this. I may even open an issue for it now while I'm thinking about it.

Anyway, the reason I mention this long-winded explanation is- displaying items in reverse order makes this issue much more prevalent. Every time you add a new item, it pushes everything else down and requires us to update offsets. With the current implementation- this would be very expensive if a user were scrolled to the end of a long chat list.

Edit: I've created issue #309 for this.

jamesbillinger commented 8 years ago

I believe I understand the issue. Absolutely positioning a row requires knowledge of the distance from the top - the height of all of the rows above it. And resizing a row at the top requires repositioning every row to the last one displayed. Doing this while also scrolling up results in janky scrolling at best.

It seemed like this might be as simple as absolutely positioning from bottom rather than top with index 0 at the bottom. However, browsers are designed to maintain scroll position based on distance from top rather than bottom, so I'm not sure how that would work. The start/stop indexes would stay the same. but what would happen to the div when content was added to the "top" of the div - absolutely positioned from the bottom?

I'm sure that wouldn't work either. Even Google Hangouts web client is janky when scrolling up. This must just be something that just doesn't work well in web.

The one thing I noticed about Hangouts though...they use relative positioning rather than absolute. I wonder if this allows them to avoid measuring each row?

Thanks again for the conversation on this. I'm not sure if this is the best place for it or if a solution is possible, but the attention is certainly appreciated.

bvaughn commented 8 years ago

Doing this while also scrolling up results in janky scrolling at best.

Yes. I can estimate the position based on the number of rows before it and their estimated size, but if the actual size differs, it can cause things to be "janky" (rows might pop in or out of visibility unexpectedly as we replace the estimates with real measurements).

It seemed like this might be as simple as absolutely positioning from bottom rather than top...

I think this would have the same UX behavior while scrolling. (If real measurements differ from estimated ones, things would look broken. The more they differ- the more they would look broken.

The one thing I noticed about Hangouts though...they use relative positioning rather than absolute. I wonder if this allows them to avoid measuring each row?

This is a possibility I haven't considered much. Maybe there's something there. I'm not sure. I think it might still be a little janky because I think we would have to essentially assume all rows (or columns) are the same size and then dynamically shift them around a bit faster (or slower) as a user scrolls based on their actual size. This might make it look like the scrolling speed is inconsistent if you have a bunch of short rows followed by a couple of really tall rows.

danieljvdm commented 8 years ago

To those of you working on chat boxes, how do you start your VirtualScroll from the bottom? I've tried scrollToIndex={this.props.messages.length - 1} but it didn't seem to work. One of my potential issues is that I'm initially keeping the box hidden in a React Bootstrap Collapse and showing it when clicked on.

jamesbillinger commented 8 years ago

We ended up going with a non-virtualized lazy-loading div. I load the most recent N messages from a conversation and then use componentDidUpdate to scroll to the bottom of the div. Then, I use a scroll event handler to fetch more messages when the user scrolls up.

The tricky part is that I have to figure out the height of the additional messages after loading more so that I can set the scrollTop back to the same message. This was simple enough in componentDidUpdate and results in surprisingly smooth scroll performance:

for (let n = 0; n < messages.conversationsData[messages.conversationIndex].messages.length - prevMessages; n++) {
  scrollTop += node.children[n].offsetHeight;
}

The nice thing as that I don't have to calculate/cache heights to render. On the other hand, if a user ever wants to scroll to the top of a very long conversation, they could experience some performance issues. It's something that I don't think most users will ever notice, and it should work nicely for us - at least until we figure out a way to virtualize.

acomito commented 7 years ago

I'm in need of an infinite load chat solution as well. Even just an easy way to have the messages list (1) start at the bottom and (2) have scrolling up trigger the loading of more messages would be nice.

bvaughn commented 7 years ago

Even just an easy way to have the messages list (1) start at the bottom and (2) have scrolling up trigger the loading of more messages would be nice.

This should be possible using the List scrollToRow prop (to auto-scroll to the last row) and InfiniteLoader (to just-in-time load more messages as a user scrolls up).

acomito commented 7 years ago

This did work: scrollToIndex ={this.props.messages.length - 1}

I'll try out InfiniteLoader for the jit loading.

kamranjon commented 6 years ago

@bvaughn do you know how I can go about triggering loadMoreRows on scroll up?

bvaughn commented 6 years ago

Should just happen automatically.

batamire commented 6 years ago

scrollToRowdoesn't work well if renderRow returns a <div> with padding or a nested <div> with padding. Scroll calculations are wrong... Any ideas?

Initial scrollToIndex works fine. Scrolling up, I don't pass scrollToIndex again, but when new message arrives I pass through scrollToIndex. This works but next scroll up always sends me back to bottom without props refreshing - back to scrollToIndex received from new message...

Using AutoSizer always requests first rows 0-x, even though I pass scrollToIndex of messageCount/bottom.

tafelito commented 6 years ago

@bvaughn I'm working on a similar case, where the loadMoreRows is not triggered. I have an infinite list showing items in reverse order (I couldn't make the list to actually reverse, might be related to #610) and the loading indicator always stays at index 0. I tried to dig on the issue, and I think the problem is because the index is always 0, even if the rowLoaded return false, the memoizer function never calls the loadMoreRows, because indexChanged is always false here https://github.com/bvaughn/react-virtualized/blob/656033edec3e33c89a468643ca861625fc5ade6f/source/utils/createCallbackMemoizer.js#L28

bvaughn commented 6 years ago

If you can provide a small repro case, I can take a look. Better yet, a PR with a failing unit test and then a fix. 😄

tafelito commented 6 years ago

I created a pen to reproduce it but I was able to make it work by using a threshold of 1. You can see the pen here if you want

https://codepen.io/tafelito/pen/bYeXmx?editors=0010

bvaughn commented 6 years ago

loadMoreRows seems to be triggered correctly for me in the Codepen you've shared. Tested in Chrome, Firefox, and Safari. (It also works if I increase the threshold property.)

tafelito commented 6 years ago

Yes, I was able to make it work, but only using the threshold=1, if you comment that line, it does not work.

Also if you preload data, in componentWillMount, loadMoreRows also gets called, even if it's not scrolled up to index=0

bvaughn commented 6 years ago

As I mentioned, it works for me even if I increase the threshold prop

tafelito commented 6 years ago

Not sure what you see, but this is what I see when not setting the threshold

nov-06-2017 20-05-47

After the initial load, when scrolling up, nothing is loading anymore

What about the preload on componentWillMount?

bvaughn commented 6 years ago

Loading data on will-mount is not a good idea b'c of upcoming React async changes. (It will be possible for will-update or will-mount to be fired without actually committing anything to the DOM, so side effects should be avoided for those methods.)

As for the bug you're seeing that I'm not, I'm not sure what to tell you. If I can't repro it (and I can't) then I can't be of much help. Try posting on Stack Overflow or somewhere similar?

tafelito commented 6 years ago

So if cWM is not the right way, what'd be the best approach to prevent an empty list when entering a screen? Any suggestions?

bvaughn commented 6 years ago

It's hard to answer this generically. I just wanted to point out that side effects like loading data aren't recommended in any of the will* lifecycle methods b'c they might be run multiple times, or aborted before actually committing anything to the DOM. Here's a cheatsheet that lists which methods are safe for side effects.

If it's possible to load the data outside of your list-rendering component, and just pass it in as a prop, this could allow you to avoid using a lifecycle method. Maybe it's possible to load an initial "page" of data outside of React entirely depending on how your application is initialized. I don't know enough about it to answer.

tafelito commented 6 years ago

Thanks @bvaughn for the quick response!

BTW, I just tested the pen on Safari, and I first though it was working without specifying the threshold but then I realized it was not using the latest version of the pen. Refreshed and now I see the same as in chrome. Just to let you know, but I understand it you can't reproduce it, I'll try to dig deeper to see if I can find anything else. Thanks anyway

baskar383 commented 4 years ago

@kof I trying the same of infinite scroll reverse order, I have tried, But i didn't get the result its keep on loading the record Down wise I have updated the code in codepen (https://codepen.io/john0075081/pen/qBExxqR), kindly refer and if you have found any solution for infinite reverse order, kindly share your code.

csnuknet commented 3 years ago

Did anyone manage to implement a reverse ordered (chat style) scroll bottom to top working? it seems the use case is popping up more so for that style of chat with few clear working examples.

Irksome having got a dynamic height top to bottom working already for another section of our app we're now struggling with a chat box that is pinned to the most recent (bottom msg) and scrolls up for the history.

jonathaneckman commented 3 years ago

I have this use case as well. An example of this would be a huge help.

amitShimon1983 commented 3 years ago

hi, do we have a solution for that case? I need to loadMoreRow on scroll up

davidivad96 commented 2 years ago

Did anyone manage to implement a reverse ordered (chat style) scroll bottom to top working? it seems the use case is popping up more so for that style of chat with few clear working examples.

Irksome having got a dynamic height top to bottom working already for another section of our app we're now struggling with a chat box that is pinned to the most recent (bottom msg) and scrolls up for the history.

After a lot of researching I found this amazing project: React Virtuoso. It's perfect for this exact use case: a reverse ordered chat style scroll bottom to top. I've been using it and it works like a charm. Take a look to this example: https://virtuoso.dev/prepend-items/

jan-wilhelm commented 2 years ago

@davidivad96 do you have any public demo showing how you use it for a chat application? Thank you!!

Bessonov commented 2 years ago

@jan-wilhelm try something like:

<Virtuoso
    data={state.contents}
    followOutput="smooth"
    alignToBottom={true}
    overscan={10000}
    itemContent={(index, message): React.ReactElement => {

But well, I experience some glitches.

devmotheg commented 2 years ago

@jan-wilhelm try something like:
<Virtuoso
  data={state.contents}
  followOutput="smooth"
  alignToBottom={true}
  overscan={10000}
  itemContent={(index, message): React.ReactElement => {
But well, I experience some glitches.

Did you manage to solve these glitches? I'm facing some too.

Bessonov commented 2 years ago

@devmotheg

Did you manage to solve these glitches? I'm facing some too.

Unfortunately, not. I reduced the amount of content, but it's rather a workaround.

bvaughn / react-virtualized

Infinite scroll in reverse order #212