Create Dictionary Vectors for certain size of `SharedStringTable` - Githubissues

polydbms / sheetreader-duckdb

MIT License

38 stars 3 forks source link

Create Dictionary Vectors for certain size of `SharedStringTable` #20

Open freddie-freeloader opened 5 months ago

freddie-freeloader commented 5 months ago

XLSX has a SharedStringTable for all unique strings in the sheets. For a certain ratio "number of string cells" / "unique string", we could create Dictionary Vectors instead of Flat Vectors.

Open questions:

[ ] How do we find out which columns have certain ratios? Maybe one column has a lot of unique string whereas another has only a few -- one should be a Flat Vector and the other a Dictionary Vector.