search

MathNya / umya-spreadsheet

A pure rust library for reading and writing spreadsheet files
MIT License
240 stars 41 forks source link

Slow to read #56

Closed KyGost closed 2 years ago

KyGost commented 2 years ago

I've just tested umya via MultiSQL by selecting 100,000 rows from a CSV (not using umya) file into an XLSX (umya), and I've found that:

Writing files is plenty fast, finished in ~1s. Reading the 100K row CSV takes >1s. Reading the 100K row XLSX takes >5m (I gave up). Pins the CPU on 1 core at a time.

Hopefully this is something we can improve.

ubamrein commented 2 years ago

Just wondering, was this on a release build or a develop build? I've experienced (e.g. with the image crate) that the oob-checks and overflow checks slow down array operations drastically. So I had like factors of 1000x improvements between release and debug builds.

KyGost commented 2 years ago

Yup, a release build.

MathNya commented 2 years ago

Improved processing speed. Please contact us if it is still slow. We will consider further improvement.

KyGost commented 2 years ago

Nice! Glad to hear. The most optimisation possible would be desired. I'll check how it performs now once I get some time and see how noticeable it is.

KyGost commented 2 years ago

Unfortunately I don't have the same dataset as I was testing with but it seems much better now.

Thanks!

KyGost commented 2 years ago

Oops... I ran the CSV one twice...

KyGost commented 2 years ago

Much better nonetheless. My test takes 1s for CSV, 6s for XLSX