If I write an HDF5 file with FESAPI 2.8, is it readable with FESAPI 1.2 ?

[corppeon]

I used v2.8 to write an HDF5 file. I used HDFView to compare the data written against v1.2 writing the same data.

I see that the object paths have changed from /RESQML/xxx to /resqml20/xxx.

Does that mean that FESAPI v1.2 will not be able to read this HDF data?

[philippeVerney]

FESAPI v1.2 should read FESAPI v2.8 HDF data transparently.

Indeed the EPC file contain XML files which explicitely contain the path to the associated HDF5 dataset. FESAPI (whatever version) reads the path in the XML of the EPC before to read the HDF5 dataset. If it does not work, it would clearly be a bug.

PS : the change in the EPC hierarchy (organization of the XML files within the EPC) is more error prone. We know that some (older) versions of some RESQML software do not succed to read this new hierarchy in EPC but latest version are fixed and can now read it. FYI, the standard itself does not mandate any particular hierarchy in the EPC, nor in the HDF5.

[ddevienne]

Hi Philippe. Can you please share light on that new EPC hierarchy?

I haven't heard about it, and despite not directly working on EPC/ETP/RESQML right now, I'd like to know about it, and see whether my code supports both the old and the new hierarchy. Also, can you share EPCs with that new hierarchy for testing? Thanks, --DD

[philippeVerney]

Hi Dominique

This new hierarchy in EPC has been decided in FESAPI to better differentiate XML from their various versions of Energistics standards (WITSML, PRODML, RESQML, EML and their numeric versions). Indeed, in all FESAPI v2.* I think, XML are now in some folders named "namespace_STANDARDNAMESTANDARDNUMERICVERSION". Regarding OPC, the standard which EPC comes from, this does not matter since the content of an EPC file is described in the root [Content_Types].xml file but if a software does not read [Content_Types].xml assuming a particular arbitrary location for the XML files then it will fail.

Here is testingPackageCpp.zip which I produce from the FESAPI example project which illustrate the current XML organization in EPC which FESAPI produces.

[ddevienne]

[...] the content of an EPC file is described in the root [Content_Types].xml file

OK, good. We are already good citizens and read that TOC, so we should be fine.

[testingPackageCpp.zip] [...] which illustrate the current XML organization in EPC which FESAPI produces.

Thanks! I'll double-check we're OK with this. I assume Laurent already knows all about this, and you're in more frequent contact with him than I am :). I appreciate the help. Cheers, --DD

PS: It all seems to work fine. Interesting new layout. I kinda like it :) Never had much exposure to non-RESQML EPCs.

D:\pdgm\data\resqml\From FESAPI 2.8>pdgm_emlx epc --info testingPackageCpp.epc
                12 ZIP folders
               305 ZIP entries
           868,221 bytes, uncompressed
           179,505 bytes, compressed (20.7%)

    <<< Per-Folder ZIP entries >>>
                 1 /
                 1 /_rels
                 2 /docProps
                 1 /docProps/_rels
                 1 /namespace_eml20
                 1 /namespace_eml20/_rels
                 6 /namespace_prodml22
                 4 /namespace_prodml22/_rels
               137 /namespace_resqml20
               137 /namespace_resqml20/_rels
                 7 /namespace_witsml21
                 7 /namespace_witsml21/_rels

D:\pdgm\data\resqml\From FESAPI 2.8>pdgm_emlx epc --stats testingPackageCpp.epc
[...]
                 1 in URI http://www.energistics.org/energyml/data/commonv2
                 1 EpcExternalPartReference

                 6 in URI http://www.energistics.org/energyml/data/prodmlv2
                 4 FluidCharacterization
                 1 FluidSystem
                 1 TimeSeriesData

               137 in URI http://www.energistics.org/energyml/data/resqmlv2
                 4 Activity
                 1 ActivityTemplate
[...]
                 1 WellboreMarkerFrameRepresentation
                 1 WellboreTrajectoryRepresentation

                 7 in URI http://www.energistics.org/energyml/data/witsmlv2
                 1 Log
                 1 Trajectory
                 1 Well
                 1 Wellbore
                 1 WellboreCompletion
                 1 WellboreGeometry
                 1 WellboreMarker

And maybe I'll had that EPC to our test suite, since it has good coverage of HDF5 ranks and datatypes:

D:\pdgm\data\resqml\From FESAPI 2.8>pdgm_emlx hdf5 --stats testingPackageCpp.h5
            16,455 Dataset stored bytes
            16,455 Dataset bytes un-chunked & un-compressed
              1.0x Compression ratio (exclude metadata overhead)

      <<< Datasets by rank >>>
               121 1D
                59 2D
                19 3D
                 6 4D

      <<< Datasets by datatype class >>>
               145 Integer
                60 Floating-Point

      <<< Datasets by rank & numeric type >>>
                 1 1D - char/bool
                 2 1D - byte
                 1 1D - short
                 4 1D - int
                92 1D - uint
                 2 1D - long
                 9 1D - ulong
                10 1D - double
                 2 2D - byte
                 2 2D - short
                 3 2D - int
                17 2D - uint
                 3 2D - long
                 1 2D - float
                31 2D - double
                 2 3D - byte
                 3 3D - ushort
                 2 3D - long
                12 3D - double
                 6 4D - double

[philippeVerney]

Hi @corppeon,

Please close this issue if the answer is OK with you. From a more general point of view, we encourage to use discourse for questions and we try to keep Github for feature requests and bugs.

[corppeon]

Thanks Philippe