Open bmschmidt opened 2 years ago
I see there is an arrow_table:
option in the prefs
but how can I format the arrow table so that it is tiled(batched) like with quadfeather?
If I pass an arrow table naively webgl buffers error out after 2**16 rows.
Edit: aha I guess you just add batch_id
column according to example here: https://github.com/nomic-ai/deepscatter/blob/901a038dc731995c15f21c4645ab9556f82517e0/integers.html#L79
I've used '.feather' as the file format here, but over the last couple years the community has standardized around '.arrow' instead, to the point where the mailing list is now discussing deprecating the 'feather' methods entirely and reminding me that "
.arrow
is the official registered extension."This will only get worse as time goes on, so will be changed in the next version bump. This is going to cause some breakage and will probably need an ability to specify, in plots, use of the '.feather' extension.