jmcnamara / excel-writer-xlsx

Perl module to create Excel XLSX files.
https://metacpan.org/pod/distribution/Excel-Writer-XLSX/lib/Excel/Writer/XLSX.pm
Other
100 stars 51 forks source link

$worksheet->write() performance #232

Closed adityagulavani closed 5 years ago

adityagulavani commented 5 years ago

First of all, thank you for this superb module.

I've a question regarding the writer's performance. I'm working with relatively small (4 - 7 MB) files. I've observed writing the output to csv files is very fast taking around 1.2s compared to writing a xlsx file which takes around 9s.

I am using write_row() to write perl arrays directly. Would it be faster to write 2d arrays with write_col()? I tried using set_optimization, but that is simply changing where the time is spent

  1. w/o set_optimization worksheet->write() 4.5s worksheet->close() 5s

  2. w/ set_optimization worksheet->write() 8s worksheet->close() 1s

The answer here might be 'writing xlsx files is inherently slow'. I want to make sure I am not missing any other performance options already available. Can we improve the performance even further?

jmcnamara commented 5 years ago

The answer here might be 'writing xlsx files is inherently slow'.

That is in fact the answer.

A CSV file is more or less the less complicated and fastest file format you could write. XLSX is a collection of inter-related XML files in a ZIP container.

The module has been optimised, almost as much as possible. There is a C version that is ~10 times faster if you'd like to try it: https://github.com/jmcnamara/libxlsxwriter

I am using write_row() to write perl arrays directly. Would it be faster to write 2d arrays with write_col()?

Not really. write_row() and write_col() are just for loops.