search

observablehq / feedback

Customer submitted bugs and feature requests
42 stars 3 forks source link

Default Data Table Cell should not slice or be more explicit about it #455

Open domoritz opened 2 years ago

domoritz commented 2 years ago

A new Data Table Cell automatically slices the data to [0,100]. However, I find this behavior problematic since it's not very obvious to a newcomer and they can think that they are looking at the full dataset. I can see that for performance reasons it may make sense to not use a full dataset but slicing to 100 seems a bit low to get a representative sample. Moreover, I would suggest making the slicing more obvious.

mbostock commented 2 years ago

This issue is not fixed, but we did increase the default slice to [0, 1000] and I am hoping we can find a way to further increase that, or remove the limit entirely. The consideration is that having no limit or too large a limit by default would lead to notebooks crashing/hanging due to performance. We will look for a way to make it work so that you can have more of a representative visual overview of your data by default. Thanks for the feedback! 🙏

domoritz commented 2 years ago

That's great. I wonder whether for some databases that can stream/lazily evaluate results, slicing can be disabled. A database client could have a flag that says that results are lazily computed. The query result could have a length property but only instantiate the rows then they are requested. Apache Arrow results already work this way.