search

mtiller / recon

Web and network friendly simulation data formats
MIT License
8 stars 4 forks source link

Dymola results file testing #9

Closed xogeny closed 10 years ago

xogeny commented 10 years ago

Convert Dymola results files into wall and meld formats and compare sizes.

Mike

xogeny commented 10 years ago

It is worth noting that the meld format (at least as it stands now), is not as efficient as the dsres file.

I'm puzzled why this is. The meld format has several advantages. First, it should be more efficient in storing string information (i.e. variable descriptions). Also, I'm avoiding redundant storage just as with the dsres file format. I could imagine uncompressed files having this issue, but even the compressed version is larger than the normal dsres file.

Clearly, this needs further analysis.

xogeny commented 10 years ago

So I looked into this, and it looks like arrays are stored as if they were documents (with "keys" interleaved between values). This is far from optimal for contiguous arrays. I need to see if there is a more efficient way to do this. I'm afraid it might involve embedding a binary string encoding of the data.