search

BenthicSubstrateMapping / PyHum

Python code to read, display, export and analyse Humminbird sidescan sonar data
Other
68 stars 28 forks source link

Installation Problems #70

Open Phi1983 opened 5 years ago

Phi1983 commented 5 years ago

Hey Daniel, I have just installed PyHum, the installation worked without errors. However, when I try python -c "import PyHum;PyHum.dotest()" I get the following error:

Traceback (most recent call last): File "", line 1, in File "/home/philipp/anaconda2/envs/pyhum/lib/python2.7/site-packages/PyHum/init.py", line 60, in from PyHum._pyhum_read import read File "/home/philipp/anaconda2/envs/pyhum/lib/python2.7/site-packages/PyHum/_pyhum_read.py", line 72, in import PyHum.pyread_single as pyread_single File "PyHum/pyread_single.pyx", line 34, in init PyHum.pyread_single File "/home/philipp/anaconda2/envs/pyhum/lib/python2.7/site-packages/PyHum/utils.py", line 37, in import dask.array as da File "/home/philipp/anaconda2/envs/pyhum/lib/python2.7/site-packages/dask/array/init.py", line 8, in from .routines import (take, choose, argwhere, where, coarsen, insert, File "/home/philipp/anaconda2/envs/pyhum/lib/python2.7/site-packages/dask/array/routines.py", line 256, in @wraps(np.matmul) File "/home/philipp/anaconda2/envs/pyhum/lib/python2.7/functools.py", line 33, in update_wrapper setattr(wrapper, attr, getattr(wrapped, attr)) AttributeError: 'numpy.ufunc' object has no attribute 'module'

I installed the program on a Ubuntu 18.04 machine within conda, using your installation instruction. Any ideas how to fix it?

Thanks Philipp

dbuscombe-usgs commented 5 years ago

Many, but no time to work on it. Please work on the issue and submit a PR. Pyhum is massively outdated and will get a radical overhaul in the winter

Phi1983 commented 5 years ago

Hey Daniel, I tried, but I'm not familiar with python. I now working on a C++ solution and I managed already to read in the Humminbird files, thanks to your File-Format description. At the moment, I stuck with the "pixel" to meter/range conversion. Do you know if there is the pulse length coded in the data files? I have also a question about your solution in the code:

theta at 3dB in the horizontal

theta3dB = np.arcsin(c/(t*(f*1000)))
#resolution of 1 sidescan pixel to nadir
ft = (np.pi/2)*(1/theta3dB) #/ (f/455)

Shouldn't the across track resolution rather depend on the pulse length than on the opening angle of the transducer?

Best Philipp

dbuscombe-usgs commented 5 years ago

There is no pulse length info in the files as far as I can tell. You are correct that it should depend on the opening angle, but the above works, so I ran with it. Please suggest an improvement if you have one.

I am also VERY interested in your C++ implementation to read the files, if you are willing to share? That part is really slow in my python/cython implementation

I have been working on a new PyHum version that works in python 3 and built on modern scalable tech such as dask, xarray, but it will take months to complete because of lack of time

Phi1983 commented 5 years ago

Hey, sorry for the late reply, I was on a cruise. I don't have an idea for improvement. I am willing to share, of course. I also wrote a new BDA which works quite well for my data.

Best, Philipp Hum_IO&BDA.zip

dbuscombe-usgs commented 5 years ago

Thanks so much for sharing this code. I will test it soon!

dbuscombe-usgs commented 5 years ago

Philipp,

Things are slow in my world - I am only now looking at your code. I have almost no experience with c++ so it may take me even longer to figure out how to use it. I will start by compiling and hopefully testing the code. Eventually, I think I would like to wrap this using cython?

What does BDA stand for? What is the purpose of the HumSIBDA.cpp code? I can see it applies filters to the sidescan data, but for what purpose and at which stage in the workflow?

Cheers,

Dan

From: Phi1983 notifications@github.com Sent: Wednesday, September 18, 2019 3:17:15 AM To: dbuscombe-usgs/PyHum Cc: Daniel David Buscombe; Comment Subject: Re: [dbuscombe-usgs/PyHum] Installation Problems (#70)

Hey, sorry for the late reply, I was on a cruise. I don't have an idea for improvement. I am willing to share, of course. I also wrote a new BDA which works quite well for my data.

Best, Philipp Hum_IO&BDA.ziphttps://github.com/dbuscombe-usgs/PyHum/files/3625812/Hum_IO.BDA.zip

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/dbuscombe-usgs/PyHum/issues/70?email_source=notifications&email_token=AA3OBXLM5EOYCANAEVFA3BLQKH52XA5CNFSM4H4TKHRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67SKEA#issuecomment-532620560, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AA3OBXKOKUWI3TVYBRSF5S3QKH52XANCNFSM4H4TKHRA.

dbuscombe-usgs commented 5 years ago

Also, please could you make the HumCoord.h file available? I believe I need that for definitions for "x2lon" and "y2lat". Thanks!

-D

From: Daniel David Buscombe Sent: Wednesday, October 16, 2019 1:55:16 PM To: dbuscombe-usgs/PyHum; dbuscombe-usgs/PyHum Cc: Comment Subject: Re: [dbuscombe-usgs/PyHum] Installation Problems (#70)

Philipp,

Things are slow in my world - I am only now looking at your code. I have almost no experience with c++ so it may take me even longer to figure out how to use it. I will start by compiling and hopefully testing the code. Eventually, I think I would like to wrap this using cython?

What does BDA stand for? What is the purpose of the HumSIBDA.cpp code? I can see it applies filters to the sidescan data, but for what purpose and at which stage in the workflow?

Cheers,

Dan

From: Phi1983 notifications@github.com Sent: Wednesday, September 18, 2019 3:17:15 AM To: dbuscombe-usgs/PyHum Cc: Daniel David Buscombe; Comment Subject: Re: [dbuscombe-usgs/PyHum] Installation Problems (#70)

Hey, sorry for the late reply, I was on a cruise. I don't have an idea for improvement. I am willing to share, of course. I also wrote a new BDA which works quite well for my data.

Best, Philipp Hum_IO&BDA.ziphttps://github.com/dbuscombe-usgs/PyHum/files/3625812/Hum_IO.BDA.zip

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/dbuscombe-usgs/PyHum/issues/70?email_source=notifications&email_token=AA3OBXLM5EOYCANAEVFA3BLQKH52XA5CNFSM4H4TKHRKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD67SKEA#issuecomment-532620560, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AA3OBXKOKUWI3TVYBRSF5S3QKH52XANCNFSM4H4TKHRA.

dbuscombe-usgs commented 5 years ago

@Phi1983 please could you share the HumCoord.h file? I'm keen to see if I can incorporate your code. Thanks

Phi1983 commented 5 years ago

Hey Daniel, sorry again for the late reply. But things are also slow in my world :( However, I managed to improve my code a little in the last weeks. It can know read the Humminbird file (from the Helix 5 series), remove the water column, make an intensity correction over range and saves the data to an ASCII File with easting northing intensity. Here are all necessary files. It requires the free library Geographic. To install the library on a ubuntu / debian machine just type: sudo apt-get install libgeographic17 you can then compile the programm with: g++ HumCoord.cpp HumIntCorrect.cpp HumIO.cpp HumSIBDA.cpp HumSI2ASCII.cpp -o HumSI2ASCII -lGeographic then run the programm with for example: ./HumSI2ASCII R00001.DAT it looks then for the necessary .SON and .IDX files in the subfolder R00001. That's the folderstructure my Humminbird stores the files.

Regarding the BDA, that's the bottom detection algorithm to remove the water column.

HumSI2ASCII.zip

Best, Philipp

dbuscombe-usgs commented 4 years ago

Excellent! Thanks so much. I got it to install on a debian based system but couldn't link to the apt version of GeographicLib, so I installed from source and updated my .bashrc so ld would find the path.

I can confirm that HumSI2ASCII also works with 998 series data. I will keep investigating but this is looking very useful and I would like to incorporate at least parts of it into the python version. May I? How should I credit you?

Phi1983 commented 4 years ago

Hey Daniel, I'm glad that the code is helping you. And nice to know that it not only works for the Helix 5. Sure you can use it.

cappelletto commented 4 years ago

Hi all, Sorry to bump this issue. I would love to give a try to PyHum but I have been facing installation issues under different setups/envs and most of them are related to libraries version (i have tried with virtual env, conda, and the git repo to no avail). So, @dbuscombe-usgs do you have any expected date for the Python 3 port? I am fully aware about how difficult it can be under time-constrained situations, so my question is more out of curiosity.

I plan to keep digging on the installation error.

@Phi1983 I am looking at you C++ implementation, and I will definitely give a try!

Cheers, Jose

dbuscombe-usgs commented 4 years ago

I am sorry to say there has been no progress with this issue. My apologies. It is still the plan, eventually, but I cannot say for sure when it gets done. I am considering 'archiving' the PyHum repo until such time that I have time to work on it again

cappelletto commented 4 years ago

It is understandable. That is a time-resource intensive task! Yet, I would like to thanks for the repository, it is really helpful to understand the underlying data format and the technique described in your article. I am trying to get some traction on using it for some post-processing pipelines. If we manage to further develop something on top of it, rest assured we will inform you.

Bests,