196 introduced a cache of the EmbeddedXlTables extracted from XLSX files to work around `openpyxl`'s slow reading of certain Excel files. There are some ways it can be improved:

[ ] "It might be worth trying caching with parquet files instead of pickles in future, they're usually lightning fast to read/write compared to pickles (although 0.8s is probably fast enough for me :) ), and the file sizes are much smaller. The downside is that you'd have to save one file per table though." (Perhaps we could wrap all tables from a file in a class?)
[ ] "Another potential issue is that the .cache/ dir gets created inside the xl2times module (i.e. beside xl2times/init.py). This seems (to me) a bit unusual. Would it be better to create it in the users' home dir (eg. pathlib.Path.home() / '.xl2times/cache)? That seems to be where most apps create their caches now - at least I seem to have lots of caches there."
[ ] Enforce a maximum cache size by evicting oldest items?
[ ] Also check file modification time to be ultra sure there are no hash collisions?

@SamRWest you're right, I wasn't sure where to put the cache directory. I'm also used to caches living in ~/.cache/ but then I wasn't sure how that translated to Windows.

I'm also used to caches living in ~/.cache/ but then I wasn't sure how that translated to Windows.

This has been standard on Windows for a while now too, which is handy. ~ ends up being c:\Users\<username> and pathlib.Path.home() is a cross-platform equivalent to ~.

Also check file modification time to be ultra sure there are no hash collisions?

If the file has been modified but the contents (and thus its content-only hash) hasn't changed, so this is probably unnecessary. The chance of an accidental collision of the file contents is infinitesimal.

Re: parquet - your pickle mechanism is actually very fast, and now I've used it for a bit, the cache is small enough that I doubt this will really become a problem, as long as we clean up old files at some point.

etsap-TIMES / xl2times

XLSX cache improvements #199

196 introduced a cache of the EmbeddedXlTables extracted from XLSX files to work around `openpyxl`'s slow reading of certain Excel files. There are some ways it can be improved:

etsap-TIMES / xl2times

XLSX cache improvements #199

196 introduced a cache of the EmbeddedXlTables extracted from XLSX files to work around openpyxl's slow reading of certain Excel files. There are some ways it can be improved:

196 introduced a cache of the EmbeddedXlTables extracted from XLSX files to work around `openpyxl`'s slow reading of certain Excel files. There are some ways it can be improved: