add ability to cancel query

cfahlgren1 / hf-data-explorer

Chrome Extension for exploring Hugging Face datasets 🔎

https://chromewebstore.google.com/detail/hugging-face-datasets-exp/algkmpgdgbindfpddilldlogcbhpkhhd

24 stars 1 forks source link

add ability to cancel query #3

Closed cfahlgren1 closed 1 week ago

cfahlgren1 commented 1 week ago

add ability to cancel long running queries

cfahlgren1 commented 1 week ago

Even with the batch reader, on very large datasets (ex FineWine), it can get OOM and kill the query. I looked at options to set recordstreamreader from arrow to a batch size other than default, but didn't see anything for apache arrow js.

It looks like the C++ API has a parameter for it, but couldn't find anything to use just yet.

A data visualizer with pagination wouldn't have this problem, but something to still think about.