search

osPlanning / omx

Open Matrix (OMX)
https://github.com/osPlanning/omx/wiki
Apache License 2.0
49 stars 18 forks source link

alphabetically based storage order based on the table names #38

Open bstabler opened 3 years ago

bstabler commented 3 years ago

from the SATURN team

One suggestion for the next revision to the specification is internal storage of the multiple tables within the OMX file. At the moment, they are stored alphabetically based on the table names so their relative position changes depending on the user-defined title. This causes some problems in implementation for our software as we use their level (ie 1st table, 2nd table, 3rd table) as the unique identifier not the table name (with the latter only optional). To get round this, we export the OMX tables (via UFM2OMX) with a Level position ‘Lxx’ prefix to the user-defined title name. When importing (via OMX2UFM), the levels are simply defined as the order they appear (ie alphabetically) and hence require some post import manipulation. The process works but it’s not as streamlined as it could be. Has anybody else flagged this as weakness?

markhudson42 commented 1 year ago

Because I've been using openmatrix a lot lately for rapid import of SATURN UFM data, I have had a trawl (as best as I can) through the openmatrix and PyTables source code and done a bit of googling around and, as far as I can tell, the limitation of tables being returned in alphabetical order of their names is built in to h5py (and maybe HDF5?) itself. This article says to me that this is just how HDF5 does it. This StackOverflow answer also says the same kind of thing.

Two years on from your comment, I am assuming that the use of the "Lnn" prefix in the UFM2OMX outputs is finalised now? I have been using this when writing my own library to manipulate the UFM2OMX outputs.

bstabler commented 1 year ago

Hi @markhudson42. I think you need to check with the saturn team - https://saturnsoftware2.co.uk/. We haven't made any revisions to OMX as a result of this suggestion.