equinor / segyio

Fast Python library for SEGY files.
Other
476 stars 214 forks source link

SegFile 's mmap() doesn't work #458

Closed kenangogogo closed 4 years ago

kenangogogo commented 4 years ago

Hi, I am using segyio to read a file about 10 Gb, and the time consumed about 151 seconds for using f.attributes(181)[:], which is too slow for my application, and i found there is a mmap() interface that can accerlerate the reading speed, however, i found the mmap() returned false in my machine, by the way, my operating system is windows 10, so is there any other solutions? any help will be appreciated.

jokva commented 4 years ago

Memory mapping is not supported on windows (it uses the posix interface), and there are no plans to do so. 151 seconds still sounds very slow for a single f.attributes call though, is this a very particular setup?

kenangogogo commented 4 years ago

Thanks for your reply, No, I just want to read the ilines through the f.attributes(181)[:], and the iline byte location is 181 in this segy file.

jokva commented 4 years ago

How do you open the file? Is the file on the local filesystem?

kenangogogo commented 4 years ago

I open the file like this, and yes, the file is on the local filesystem. f = segyio.open(segy_file_path, iline=iline_byte_loc, xline=xline_byte_loc, ignore_geometry=True) and then, i get the ilines using the following code: ilines = f.attributes(iline_byte_loc)[:] (this line consume most of the time in my case)

jokva commented 4 years ago

It should certainly take up the most of the time in your program, but your code looks fine, and there is no reason for it to be that slow. Without being able to run tests on your system there's unfortunately little I can do for you. Are other file operations slow?

kenangogogo commented 4 years ago

thank you all the same, other file operations seems ok, maybe there is something wrong with my hardware, i'll run some tests on other computer to see if existing same problem.