Panics and strange behavior when using > 2 million rows in custom ListView model

[qarmin]

Slint 1.2.2

When trying to add 10 millions rows to model, after scrolling down to ~5.6 million record I got crash

Looks that after 2/3 millions of items it creates empty space between items

Scroll bar after 3.3 million record go back to ~2.8 million record, probably due adding space after items

Selection dissapears when selected row is not in screen - this happens even with 10 rows

Project to test - czkawka_slint_gui.zip

Video with visible problems:

https://github.com/slint-ui/slint/assets/41945903/d146dc8a-68ec-46bc-8a63-8be2169b22c9

Panic

thread 'main' panicked at /home/rafal/Projekty/Rust/czkawka/target/debug/build/czkawka_slint-0d08bfe5f757d603/out/main_window.rs:4492:141:
called `Option::unwrap()` on a `None` value
stack backtrace:
   0: rust_begin_unwind
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/panicking.rs:595:5
   1: core::panicking::panic_fmt
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/panicking.rs:67:14
   2: core::panicking::panic
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/panicking.rs:117:5
   3: core::option::Option<T>::unwrap
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/option.rs:935:21
   4: czkawka_slint::slint_generatedMainWindow::InnerSelectableTableView_root_27::subtree_component
             at /home/rafal/Projekty/Rust/czkawka/target/debug/build/czkawka_slint-0d08bfe5f757d603/out/main_window.rs:4492:90
   5: czkawka_slint::slint_generatedMainWindow::InnerComponent_selectabletableview_8::subtree_component
             at /home/rafal/Projekty/Rust/czkawka/target/debug/build/czkawka_slint-0d08bfe5f757d603/out/main_window.rs:9533:22
   6: <czkawka_slint::slint_generatedMainWindow::InnerComponent_selectabletableview_8 as i_slint_core::component::Component_vtable_mod::Component>::get_subtree_component
             at /home/rafal/Projekty/Rust/czkawka/target/debug/build/czkawka_slint-0d08bfe5f757d603/out/main_window.rs:9799:14
   7: <czkawka_slint::slint_generatedMainWindow::InnerComponent_selectabletableview_8 as const_field_offset::PinnedDrop>::drop::VT::get_subtree_component
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-core-1.2.2/component.rs:40:1
   8: i_slint_core::component::Component_vtable_mod::ComponentTO::get_subtree_component
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-core-1.2.2/component.rs:40:1
   9: i_slint_core::item_tree::ItemRc::find_sibling
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-core-1.2.2/item_tree.rs:280:21
  10: i_slint_core::accessibility::accessible_descendents::{{closure}}
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-core-1.2.2/accessibility.rs:57:39
  11: <core::iter::sources::from_fn::FromFn<F> as core::iter::traits::iterator::Iterator>::next
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/iter/sources/from_fn.rs:69:9
  12: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::next
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/iter/adapters/map.rs:103:9
  13: alloc::vec::Vec<T,A>::extend_desugared
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/alloc/src/vec/mod.rs:2796:35
  14: <alloc::vec::Vec<T,A> as alloc::vec::spec_extend::SpecExtend<T,I>>::spec_extend
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/alloc/src/vec/spec_extend.rs:17:9
  15: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/alloc/src/vec/spec_from_iter_nested.rs:43:9
  16: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/alloc/src/vec/spec_from_iter.rs:33:9
  17: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/alloc/src/vec/mod.rs:2696:9
  18: core::iter::traits::iterator::Iterator::collect
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/iter/traits/iterator.rs:2053:9
  19: i_slint_backend_winit::accesskit::AccessKitAdapter::build_node_for_item_recursively
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-backend-winit-1.2.2/accesskit.rs:235:14
  20: i_slint_backend_winit::accesskit::AccessKitAdapter::build_new_tree::{{closure}}
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-backend-winit-1.2.2/accesskit.rs:272:13
  21: i_slint_core::properties::CURRENT_BINDING::<impl i_slint_core::properties::CURRENT_BINDING>::set
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/scoped-tls-hkt-0.1.4/src/lib.rs:265:25
  22: i_slint_core::properties::PropertyTracker<DirtyHandler>::evaluate_as_dependency_root
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-core-1.2.2/properties.rs:1295:17
  23: i_slint_backend_winit::accesskit::AccessKitAdapter::build_new_tree
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-backend-winit-1.2.2/accesskit.rs:271:23
  24: i_slint_backend_winit::accesskit::AccessKitAdapter::register_component::{{closure}}::{{closure}}
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-backend-winit-1.2.2/accesskit.rs:133:49
  25: accesskit_winit::platform_impl::platform::Adapter::update_if_active
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/accesskit_winit-0.14.4/src/platform_impl/unix.rs:45:28
  26: accesskit_winit::Adapter::update_if_active
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/accesskit_winit-0.14.4/src/lib.rs:245:9
  27: i_slint_backend_winit::accesskit::AccessKitAdapter::register_component::{{closure}}
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-backend-winit-1.2.2/accesskit.rs:133:29
  28: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/ops/function.rs:250:5
  29: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/alloc/src/boxed.rs:2007:9
  30: i_slint_core::timers::TimerList::maybe_activate_timers::{{closure}}
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-core-1.2.2/timers.rs:272:29
  31: std::thread::local::LocalKey<T>::try_with
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/thread/local.rs:270:16
  32: std::thread::local::LocalKey<T>::with
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/thread/local.rs:246:9
  33: i_slint_core::timers::TimerList::maybe_activate_timers
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-core-1.2.2/timers.rs:230:24
  34: i_slint_backend_winit::event_loop::run::{{closure}}
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-backend-winit-1.2.2/event_loop.rs:591:17
  35: i_slint_backend_winit::event_loop::run::{{closure}}::{{closure}}
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-backend-winit-1.2.2/event_loop.rs:658:65
  36: i_slint_backend_winit::event_loop::CURRENT_WINDOW_TARGET::<impl i_slint_backend_winit::event_loop::CURRENT_WINDOW_TARGET>::set
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/scoped-tls-hkt-0.1.4/src/lib.rs:265:25
  37: i_slint_backend_winit::event_loop::run::{{closure}}
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-backend-winit-1.2.2/event_loop.rs:658:17
  38: winit::platform_impl::platform::sticky_exit_callback
  39: winit::platform_impl::platform::x11::EventLoop<T>::run_return::single_iteration
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/x11/mod.rs:324:13
  40: winit::platform_impl::platform::x11::EventLoop<T>::run_return
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/x11/mod.rs:483:27
  41: winit::platform_impl::platform::EventLoop<T>::run_return
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/mod.rs:785:56
  42: <winit::event_loop::EventLoop<T> as winit::platform::run_return::EventLoopExtRunReturn>::run_return
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform/run_return.rs:51:9
  43: i_slint_backend_winit::event_loop::run
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-backend-winit-1.2.2/event_loop.rs:649:20
  44: <i_slint_backend_winit::Backend as i_slint_core::platform::Platform>::run_event_loop
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-backend-winit-1.2.2/lib.rs:231:9
  45: slint::run_event_loop::{{closure}}
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/slint-1.2.2/lib.rs:227:49
  46: i_slint_core::with_platform::{{closure}}
  47: std::thread::local::LocalKey<T>::try_with
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/thread/local.rs:270:16
  48: std::thread::local::LocalKey<T>::with
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/std/src/thread/local.rs:246:9
  49: i_slint_core::with_platform
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-core-1.2.2/lib.rs:99:33
  50: i_slint_backend_selector::with_platform
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/i-slint-backend-selector-1.2.2/lib.rs:98:5
  51: slint::run_event_loop
             at /home/rafal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/slint-1.2.2/lib.rs:227:5
  52: <czkawka_slint::slint_generatedMainWindow::MainWindow as i_slint_core::api::ComponentHandle>::run
             at /home/rafal/Projekty/Rust/czkawka/target/debug/build/czkawka_slint-0d08bfe5f757d603/out/main_window.rs:10163:14
  53: czkawka_slint::main
             at ./src/main.rs:20:5
  54: core::ops::function::FnOnce::call_once
             at /rustc/cc66ad468955717ab92600c770da8c1601a4ff33/library/core/src/ops/function.rs:250:5

[ogoffart]

Thanks for your bug report. I can reproduce the problem.

One problem is that we use f32 for the coordinate, and that the content-y of the underlying ListView can become really big, so big that we get rounding error on the f32 computations when computing the location of the items within the viewport.

In particular the problem is there: https://github.com/slint-ui/slint/blob/6cd68e84a7be8a43baa060565f0bbe2a11b9101b/internal/core/model.rs#L1081

the size of the row is 42.0px. So y in that expression, which is the position in pixel within the viewport, is going to be 42 * some_index, where some_index can be millions. So we are way over 100M pixels The listview_layout function will position the element at y , and add 42px to it. The problem is that in f32, y+42 is actually a few pixel off as that number might not be representable in a f32. hence the wrong gaps.

I don't explain the panic yet. It tries to query the vector of instance with a completely wrong index, as it iterate over widgets in the accessibility code.

[ogoffart]

The panics happen because the view is relayouted between the range check and the call to get_subtree:

https://github.com/slint-ui/slint/blob/6cd68e84a7be8a43baa060565f0bbe2a11b9101b/internal/core/item_tree.rs#L429-L439

We query the range line 429, and check the validity of index line 432, but the get_subtree notice that the tree is dirty (not sure why) and redo the layout. And the item computation turns out to be something different because the floating point computation give different result, and we just query the components outside of the range. (next_subtree_index is outside of the new range which is recomputed by get_subtree itself)

[ogoffart]

https://github.com/slint-ui/slint/pull/3709 is fixing the panic.

[ogoffart]

3709 fixed one panic, but there is still one panic reported in https://github.com/slint-ui/slint/pull/3709#issuecomment-1771498412:

thread 'main' panicked at /home/rafal/Projekty/Rust/czkawka/target/debug/build/czkawka_slint-3baf4a5d90d5188c/out/main_window.rs:6791:159:
called `Option::unwrap()` on a `None` value
stack backtrace:
   4: czkawka_slint::slint_generatedMainWindow::InnerComponent_text_54::item_geometry
   5: <czkawka_slint::slint_generatedMainWindow::InnerComponent_text_54 as i_slint_core::item_tree::ItemTree_vtable_mod::ItemTree>::item_geometry
   6: <czkawka_slint::slint_generatedMainWindow::InnerComponent_text_54 as const_field_offset::PinnedDrop>::drop::VT::item_geometry
   7: i_slint_core::item_tree::ItemTree_vtable_mod::ItemTreeTO::item_geometry
   8: i_slint_core::item_tree::ItemRc::geometry
   9: i_slint_backend_winit::accesskit::AccessKitAdapter::build_node_without_children
  10: i_slint_backend_winit::accesskit::AccessKitAdapter::build_node_for_item_recursively::{{closure}}
  11: i_slint_core::properties::CURRENT_BINDING::<impl i_slint_core::properties::CURRENT_BINDING>::set
  12: i_slint_core::properties::PropertyTracker<DirtyHandler>::evaluate_as_dependency_root
  13: i_slint_core::properties::PropertyTracker<DirtyHandler>::evaluate
  14: i_slint_backend_winit::accesskit::AccessKitAdapter::build_node_for_item_recursively
  15: i_slint_backend_winit::accesskit::AccessKitAdapter::build_node_for_item_recursively::{{closure}}

Which is just another symptom of https://github.com/slint-ui/slint/issues/3464 as the accessibility iterate over items, but while iterating the listview relayout code is executed that produces different result and delete the parent element.

---

We have a combination of several problem:

1. The main issue is that we represent coordinate in f32, and the maximum integer that can fit in a f32 is 16777216. But the coordinate of the viewport relative to the flickable can be much larger than that when having many millions entries. And therefore their position is not accurately representable.

2. We do relayout the list view more often than needed. Commit 4d0568873069b7ca7134a5e3f18c68ff3a1f54e9 actually removed the listview_geometry_tracker.evaluate_if_dirty which would only relayout when the listview is dirty. So we relayout all the time when traversing the elements, which is a lot. Fortunately, the relayout is fairly fast as there are never so many elements visible at the same time. But still.

3. If you sum it up, we get the layout redone too often wand because of point 1. it result in different amount of element which result in item being deleted which cause a panic because of #3464

Also the fact that accessibility is rebuilding the tree even if there is no accessibility backend which is a bug in accesskit.

What can we do?

First thought was to use f64 for coordinates on the desktop. We already have a switch in https://github.com/slint-ui/slint/blob/5bf2c7192b6264947e5e80e7e85f831f4ea3f852/internal/core/lib.rs#L82 for the MCUs, we could as well allow to switch to f64. But it turns out that even doing so wouldn't help fully because the femtovg and skia renderer use f32 internally and we do translate the canvas when we get into items. Instead of translating the canvas like this, we could maintain offsets in f64 within slint, but that's some extra refactoring.

Another possibility would be to not have the listview items as chilren of the viewport, but instead, have them as sibling of the viewport. And so the layouting code can just give them a y position relative to the ListView instead of relative to the viewport. I think this is possible, but require some refactoring in the runtime and compiler.

[Enyium]

Couldn't this be fixed with what I hinted at in #5758 (some modulo-like calculation), instead of letting a number grow uncontrollably? Scroll deltas should always be whole physical pixels, right? Then you should actually be able to maintain the illusion of a continuous list, while a rendered-items block is regularly shifted during scrolling (its data together with its position), without getting graphical-rendering imperfections that suddenly show on content block shift. I'm not sure whether these mid-animation shifts would require extra support from the animation system.

Of course, there's still the issue of how things are communicated to the user. For my needs in the linked discussion, the component would communicate a duration (a 64-bit value) that the content block should start with; meaning, the backend would provide appropriate data that the component then renders. This would be my precise "actual position in the data". For cases with an overly long list with a scrollbar, you could also establish a callback that tells the user to adapt the window into the data that the backing model then would represent (using integer indices). This means, you'd have the actual model, and an integer pair of model start in the virtual data and virtual-model length.

CC: @ezschemi