[BUG] React virtuoso cannot render more than 800k lines

[khelif96]

Describe the bug I've been unable to get React Virtuoso to render a list with 1 million elements or more. It often caps out at ~ 800,000ish lines. This does not work on either the GroupedVirtuoso component or the normal Virtuoso component.

React window also faces this same problem https://github.com/bvaughn/react-window/issues/108 React virtualized does not have this limitation and i've been able to get it to load any size list.

The application i am building requires being able to load any arbitrary size list of elements. I would have loved to use React Virtuoso but unfortunately this limitation is the biggest dealbreaker. I just wanted to put it on your radar in case this has not been reported before.

Reproduction https://codesandbox.io/s/boring-framework-7iodep?file=/src/App.tsx

To Reproduce Steps to reproduce the behavior:

1. Go to above codesandbox link.
2. Try to scroll to the end of the list
3. See that you are unable to scroll to the 1 millionth element.
4. Uncomment GroupedVirtuoso component and observe the same behavior.

Expected behavior I would expect React Virtuoso to be able to render any size list.

Screenshots

Desktop (please complete the following information):

• Browser: Chrome, Firefox

[petyosi]

I am aware of that limitation, there are a few open issues that mention it. It's solvable, but it needs some effort on my side. Would you or your employer consider sponsoring the development of that capability? I'm also open to PRs.

[khelif96]

I would love to take a look would you mind pointing me in the right direction / or estimating the level of effort you would think this would take to fix ?

[petyosi]

The gist of it is that you need to start multiplying the scroll delta and increase the internal offset accordingly. As far as I remember, react-virtualized was doing something similar. Take a look around the source code to see if you're comfortable with it, also check https://urx.virtuoso.dev/ for docs on how the state management is done. I would say it's a few days of work, testing included.

[oleggrishechkin]

DOM elements have cap height/width, and anything more than this cap will be hidden. I think you can use something like infinite scroll (where elements outside visible area will be collapsed) or you should use pagination. It's not a vitrual scroll, but a possible solution to render so much elements as you want.

[khelif96]

@oleggrishechkin I'm a little unsure about what you mean I attempted to change the size of the elements in my demo and i was unable to bypass 800,000 ish limit which indicates the limitation is in the library and not caused by a hard DOM height cap since smaller elements should equate to a lower total dom height and should allow us to render more elements.

For our application it's not feasible to use pagination since we require a continuous scrollable feed of data.

[khelif96]

@petyosi could we reopen this issue since it does seem to be a limitation in the library?

Out of curiosity what would sponsorship of this improvement look like?

[oleggrishechkin]

@khelif96 I check your codesendbox from this topic and react-virtuoso CAN render 1000000+ elements with 20px element height.

Do you try to set defaultItemHeight to some very low value like 5,4,3,2.... For me chrome cap for item height is ~16kk, so your item height should be lower than 16kk / count. I tried defaultItemHeight={5} and virtuoso rendered all 1kk elements.

[petyosi]

@khelif96, if you're interested in sponsoring this, ping me over email - we can talk. I don't find it useful to keep open issues for things that I am unlikely to work on shortly. The potential scope of the component is infinite - no point in tracking that.