search

tbeu / matio

MATLAB MAT File I/O Library
https://matio.sourceforge.io
BSD 2-Clause "Simplified" License
330 stars 97 forks source link

Performance when reading large .mat file with nested structs #163

Closed aitorperezzz closed 3 years ago

aitorperezzz commented 3 years ago

Hi, I am trying to read a large .mat file (approximately one gig) on Linux and I am seeing that accessing variables inside nested structs takes a long time (about 15 seconds when the nesting level is around 3). Is this to be expected? Is there any way to improve performance?

I can provide versions of the library, compiler, os and general environment if needed.

tbeu commented 3 years ago

See also #55, #65 and #140. I am afraid it is the best we can achieve currently.

Can you run

/usr/bin/time -f "%es %MK" ./tools/matdump file.mat > log1.txt
/usr/bin/time -f "%es %MK" ./tools/matdump -d file.mat > log2.txt

and provide the stdout. If you have a simple m script to create such a MAT file in MATLAB I would be interested, too.

I also created https://github.com/tbeu/matioPerformance (on Win OS only) to compare MAT file reading, but it is not up to date regarding the version of libmatio.dll.

tbeu commented 3 years ago

Closing as duplicate of #65.

tbeu commented 8 months ago

@aitorperezzz FYI: https://github.com/tbeu/matioPerformance is updated to the upcoming libmatio v1.5.24.