-
(Updated on 9th Nov)
To provide ordered data from a data frame potentially storing large volume of data, efficient sorting capability needs to be built. It can be built by providing `sort()` and `a…
-
Can you provide some details on the design o how inverted index search would be implemented with the columnar format .
Also what kind of feature store use cases are suited for lanceDB , typically i …
-
when storing a json, it would be beneficial to store it in a columnar format so each indexed field is treated like a column in a regular table.
then Pinot can possibly be able to also use different …
-
### Backend
VL (Velox)
### Bug description
We are currently working on integrating Gluten into our WatsonX.Data's Spark environment. However, after enabling Gluten and running the TPCH benchm…
-
### Bug Description:
I have 7 columnar indexes but there is high **unaccounted memory usage**:
**Image 1:**
Below are the status of indexes using memory and memory mapped to disk. which is very…
-
### Component(s)
exporter/file
### Is your feature request related to a problem? Please describe.
Parquet Format:
Parquet is a columnar storage file format optimized for big data processing framew…
-
I'm wanting to look up the terms for a given document, that is `Document -> Field -> Terms`. Something like what term vectors provide in Lucene. (However, I see positions are already [stored a little …
-
### What's wrong?
H! We have two separate tablespaces: default(ssd with relatively small capacity for current data) and hdd (high capacity, historical data) and don't want to store columnar tables on…
-
This would minimize contortions for casting between various index sizes.
See: https://github.com/rust-lang/rust/issues/89193#issuecomment-92627025
> Since there is a bit of chatter here on the use…
-
**Summary**
To enable data compression for rapid engine. and to support data scanning on compressed data block without uncompressing the data.
```
"Lightweight Indexing on Compressed Data"
aut…