search

mahalex / MatFileHandler

MATLAB .mat-file reading/writing API for .NET Standard
MIT License
40 stars 12 forks source link

Compression #3

Closed bernie678 closed 5 years ago

bernie678 commented 5 years ago

It seems the output files are always compressed. Is it possible to write them uncompressed.

Part of my concern is speed but I have also noticed and issue with increasing memory while writing.

For one example, I read in a 300 Mbyte csv file (1250 columns, 7500 rows) and ty to write it as a MatLab struct. The memory steadily increases until it bombs with over 10Gbytes of memory allocated (per task manager.) I have not debugged into the MatFileHandler code but I am guessing this might be part of the process of building the compressed image in memory.

On the other hand, I realize the memory issue could be completely separate from the compression process.

Any advice appreciated.

mahalex commented 5 years ago

I just published a new release (1.2.0) that makes it possible to write uncompressed files. You can try it and see if it helps. Meanwhile, I'll still try to investigate the memory allocation issue.

bernie678 commented 5 years ago

Great. Thanks.