TGSAI / mdio-python

Cloud native, scalable storage engine for various types of energy data.
https://mdio.dev/
Apache License 2.0
37 stars 13 forks source link

SEG-Y export not preserving order for specific shot data #464

Open tasansal opened 3 weeks ago

tasansal commented 3 weeks ago

For non-public SEG-Y (6733_5DPREPNODMLT) shot data with selection mask, the order of the file is wrong with the default new_chunks used in export. Works fine with (4, 4, 137, 6, 2851) but that's very large chunks. Need to investigate why.

ingestion

segy_to_mdio(
    “segy_path”: ...,
    “mdio_path_or_buffer”: ...,
    “index_names”: ["inline","xline","offset","azimuth"],
    “index_bytes”: [213,217,37,197],
    “index_types”: ["int32","int32","int32","ibm32"]
    “chunksize”: [8,32,1,1024],
)

export

mdio_to_segy(
    mdio_path_or_buffer=...,
    output_segy_path=...,
    selection_mask=cut_mask,
    access_pattern="01234",
    # new_chunks=(4, 4, 137, 6, 2851), # these chunks work
    client=None
)

Example

MASK          DATA
0 0 0 0 0     0 0 0 0 0
0 1 0 1 0     0 a 0 b 0  
0 1 0 1 0     0 c 0 d 0
1 1 1 1 1     e f g h i
Mask order      0 1 2 3 4 5 6 7 8
Expected order  a b c d e f g h i
Actual order    a c b d e f g h i