search

equinor / segyio

Fast Python library for SEGY files.
Other
476 stars 214 forks source link

Agnostic Text Header Reading #451

Closed GGDRriedel closed 4 years ago

GGDRriedel commented 4 years ago

I work with a lot of "historic" data and the text headers from the 60s and 70s sometimes contain some really weird stuff.

Now I have several text headers that for some reason contain "\x04" in the first line.

The segyfile.text[0] looks like this as a raw string coming out of the file:

bytearray(b'C01 CLIENT: xxxxr COMPANY: xxxxxx \x04')

and stop after that since apparently in EBCDIC x04 is the strg terminator and in ASCII it's the End of Transmission.

Agnostic "viewers" hoewever show me the full header, despite there being this sign because they do not really use the information insed the string but rather treat is as a "blob" and decode it no matter what comes.

Is there a way of getting some "agnistic" text?

jokva commented 4 years ago

Interesting.

So the C functions really only read the first 3200 bytes, and map them into EBCDIC (this was a mistake, but maybe I get some time to fix that soon). It is not aware of any end-of-file, end-of-transmission or other special characters, it's just a 256-character translation table.

What I notice it does do internally is a PyByteArray_FromStringAndSize, after doing strlen on it. My guess it's not the \x04, it's a null byte after. I think this is a bug in segyio.

Are you comfortable compiling and testing a patch yourself, or should I make a special build for you with a fix?

GGDRriedel commented 4 years ago

From a personal standpoint I would love to do it myself but I am not deep enough into it to be able to do that , just an... educated user? I would be really glad if you could do it. Unfortunately I can not share the segy for testing purposes :(

jokva commented 4 years ago

No problem, I'll get a build up with a patch applied soon-ish.

jokva commented 4 years ago

The 1.9.1-rc1 release is being uploaded as we speak, with the patch applied.

Install it with pip install segyio==1.9.1-rc1, and please see if it works for you.

GGDRriedel commented 4 years ago

couldn't find rc1 but pip suggested 1.9.1a1, took that and it works! Thanks a lot!

jokva commented 4 years ago

You're right, there was a mismatch between the tag message and the tag itself. a1 is correct, and if it works for you I'm happy, and it's to-be mainline.