search

jlolling / talendcomp_tFileExcel

Talend components tFileExcel* to read and write Excel documents
Apache License 2.0
14 stars 7 forks source link

Java Heap Space #1

Closed TechWatching closed 6 years ago

TechWatching commented 7 years ago

Do these components handle the memory issues that can be encountered with the native Excel components (OutOfMemoryException) ?

I am particularly interested by the scenario where data is written in several worksheets. What happens with the tFileOutputExcel component when you want to write data in 2 worksheets is that you use 2 tFileOutputExcel (one for each sheet), the first works correctly but the second load the entire excel document in memory which causes the java heap space error.

jlolling commented 7 years ago

Hi, no there is no way to handle such problems within the component. I am working on a streaming mode also for reading files. The problem is, reading a sheet alone is not possible. All components (also mine and Talend) have to read the file at once and can now selectively output the data from a sheet but all nodes are within the memory. There are some ideas to reduce the data for not needed sheets but this is not implemented yet.