search

Textualize / textual

The lean application framework for Python. Build sophisticated user interfaces with a simple Python API. Run your apps in the terminal and a web browser.
https://textual.textualize.io/
MIT License
25.08k stars 767 forks source link

DataTable "view" of a larger data set #1892

Open multimeric opened 1 year ago

multimeric commented 1 year ago

The DataTable seems to have performance issues with about a million cells. If we reduce the number of cells in memory at one time to a fraction of that, and then lazily load new rows as the user scrolls down, plus delete old rows, it seems like we might solve these performance issues. Not entirely sure though because I'm not sure how the implementation works.

The simplest implementation of this might be via a pagination mechanism where the user has to implement get_chunk() -> Iterable[Row] and the class handles the loading as the user approaches the end of the current chunk . This would also support viewing external data such as in a database or a remote API and avoid having to load everything into memory.

github-actions[bot] commented 1 year ago

Thank you for your issue. Give us a little time to review it.

PS. You might want to check the FAQ if you haven't done so already.

This is an automated reply, generated by FAQtory

multimeric commented 11 months ago

I note that "Lazy loading API" for DataTable is listed in the Roadmap. I might be able to give implementation a go if one of the maintainers is interested.

kdkavanagh commented 5 months ago

Anyone come up with an implementation/workaround for this? Very interested as well

max-arnold commented 5 months ago

Could this be used as an interim solution? https://github.com/tconbeer/textual-fastdatatable