search

CameronBodine / PINGMapper

Open-source interface for processing recreation-grade side scan sonar datasets and reproducibly mapping benthic habitat
https://cameronbodine.github.io/PINGMapper/
MIT License
39 stars 3 forks source link

Mosaic options to exclude specific chunks #103

Closed dfcscience closed 4 months ago

dfcscience commented 8 months ago

Hi Cameron - First, thanks so much for creating and maintaing PINGMapper. It's a really great piece of software.

As you know, I have a Humminbird mounted on my little autonomous boogie board. I start recording before it starts and auto mission and I stop it when it gets back. With this setup, I can't stop recording in the turns. After a lot of trial and error, I developed a method that helps me get better looking imagery. I modified my auto missions to include a waiting period before it starts a transect line, so the last turn is not smeared into the beginning of the transect. When processing in main.py, I set nchunk to a lower value, which results in more individual georectified images. This allows me to simply exclude the images from the turns in my final map.

Would it be possible to run PINGMapper to create the individual georectified images? After I view them, I can create a list of chunks to exclude in the mosaicking. Would it be possible to run it in 2 separate steps like this?

Thanks again!

CameronBodine commented 8 months ago

Hi Dan, I'm glad you are pleased with PINGMapper so far. I have another user doing autonomous surveys and is encountering similar issues with not being able to stop the recording. I plan on an automated approach which would allow user-specified maximum heading deviation over a distance to flag chunks for removal from the final mosaic. I also like your suggestion but am away from my computer this week. I think there is something you can try.

In the utils folder is a script called main_mosaic_transects.py that may work for you. It is designed to mosaic overlapping recordings. First, you would process the recording as usual using the main.py script. Choose to export rectified imagery but don't mosaic. After the script runs, go through the text folders in each of the port and start folders and remove the chunks which should not be included in the final mosaic. Then point the main_mosaic_transects.py at the project folder, make sure the rect_* parameters are set to False and set the mosaic parameters are true and try running the script. Hopefully it will work. Let me know how it goes and report any issues.

Cam

dfcscience commented 8 months ago

Hi Cam, Thanks for this. I think we're headed in the right direction. I followed your instructions above. I removed the chunks from the rect_wcr and rect_wcp (both star and port), and pointed main_mosaictransects.py to the processed data folder (I assume that's what you mean by project folder). I set the rect* parameters to False and the mosaic parameters to True. The script runs (I made sure to activate my ping virtual environment) but there's no output. Any idea what I'm doing wrong?

I have copied the part of main_mosaic_transects.py that I modified below

transectDir = r'C:\Users\HEREON\Documents\MyPythonRepos\PINGMapper\procData\AciTrezza72'
projName = 'AciTrezza72_Mosaic'

# Rectification Parameters
rect_wcp = False #Export rectified tiles with water column present
rect_wcr = False #Export rectified tiles with water column removed/slant range corrected

# Mosaic Parameters
mosaic_transect = 1 #Export rectified tile mosaic; 0==Don't Mosaic; 1==Do Mosaic - GTiff; 2==Do Mosaic - VRT
mosaic_all_transects = True # True: Mosaic transects into one; False: Don't Mosaic
resampleAlg = 'cubic' # mode, average, gauss, lanczos, bilinear, cubic, cubicspline, nearest
pix_fn = 'average'
overview = Truezza72_Mosaic'

# Rectification Parameters
rect_wcp = False #Export rectified tiles with water column present
rect_wcr = False #Export rectified tiles with water column removed/slant range corrected

# Mosaic Parameters
mosaic_transect = 1 #Export rectified tile mosaic; 0==Don't Mosaic; 1==Do Mosaic - GTiff; 2==Do Mosaic - VRT
mosaic_all_transects = True # True: Mosaic transects into one; False: Don't Mosaic
resampleAlg = 'cubic' # mode, average, gauss, lanczos, bilinear, cubic, cubicspline, nearest
pix_fn = 'average'
overview = True
CameronBodine commented 8 months ago

Thanks for the info. Can you send me the logs in the AciTrezza72_Mosaic folder?

dfcscience commented 8 months ago

The folder is empty. I can't seem to copy the text in the windows terminal. The only log file is from the original processing in the logs folder

CameronBodine commented 8 months ago

I made one slight change. To update, run git stash then git pull in the console.

In your case, the output will be in C:\Users\HEREON\Documents\MyPythonRepos\PINGMapper\procData\AciTrezza72\sonar_mosaic folder, not in C:\Users\HEREON\Documents\MyPythonRepos\PINGMapper\procData\AciTrezza72\AciTrezza72_Mosaic.

Let me know!

dfcscience commented 8 months ago

Success! side_scan_mosaic

CameronBodine commented 8 months ago

Awesome! And what a beautiful spot. I envy those who in clear water!

dfcscience commented 4 months ago

Hi Cam- I just tried to run this again, but it throws an error. I replicated the error on both my macbook and my windows PC. I copied the output from my mac below. The csv file that it's looking for is in the meta folder.

Any ideas? Thanks in advance!

Daniels-MacBook-Pro-3:PINGMapper dfc$ python /Users/dfc/Documents/MyPythonRepos/PINGMapper/utils/main_mosaic_transects.py Version: 2.16.1 Eager mode: True

Smoothing tracklines... [Parallel(n_jobs=1)]: Done 0 tasks | elapsed: 0.0s Traceback (most recent call last): File "/Users/dfc/Documents/MyPythonRepos/PINGMapper/utils/main_mosaic_transects.py", line 315, in Parallel(n_jobs= np.min([len(portstar), threadCnt]), verbose=10)(delayed(smoothTrackline)(sons) for sons in portstar) File "/opt/anaconda3/envs/ping/lib/python3.11/site-packages/joblib/parallel.py", line 1918, in call return output if self.return_generator else list(output) ^^^^^^^^^^^^ File "/opt/anaconda3/envs/ping/lib/python3.11/site-packages/joblib/parallel.py", line 1847, in _get_sequential_output res = func(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Users/dfc/Documents/MyPythonRepos/PINGMapper/utils/main_mosaic_transects.py", line 109, in smoothTrackline son._loadSonMeta() # Load ping metadata ^^^^^^^^^^^^^^^^^^ File "/Users/dfc/Documents/MyPythonRepos/PINGMapper/src/class_sonObj.py", line 1937, in _loadSonMeta meta = pd.read_csv(self.sonMetaFile) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/ping/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 1026, in read_csv return _read(filepath_or_buffer, kwds) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/ping/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 620, in _read parser = TextFileReader(filepath_or_buffer, kwds) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/ping/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 1620, in init self._engine = self._make_engine(f, self.engine) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/anaconda3/envs/ping/lib/python3.11/site-packages/pandas/io/parsers/readers.py", line 1880, in _make_engine self.handles = get_handle( ^^^^^^^^^^^ File "/opt/anaconda3/envs/ping/lib/python3.11/site-packages/pandas/io/common.py", line 873, in get_handle handle = open( ^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'C:/Users/HEREON/Documents/MyPythonRepos/PINGMapper/procData/AciTrezz69\meta\B002_ss_port_meta.csv'

CameronBodine commented 4 months ago

Is this issue isolated to just this recording or does it happen with others as well?

Please try reprocessing this recording from scratch and try running this script again from the same computer. If it persists, can you send me the recording?

dfcscience commented 4 months ago

I had the issue with several recordings. Reprocessing appears to have solved the problem

On Thu, Apr 18, 2024 at 1:10 PM Cameron Bodine @.***> wrote:

Is this issue isolated to just this recording or does it happen with others as well?

Please try reprocessing this recording from scratch and try running this script again from the same computer. If it persists, can you send me the recording?

— Reply to this email directly, view it on GitHub https://github.com/CameronBodine/PINGMapper/issues/103#issuecomment-2063611766, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJNX7H5X5IAIQQ2IDEUWQRDY56SYPAVCNFSM6AAAAABAS7OQQWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRTGYYTCNZWGY . You are receiving this because you authored the thread.Message ID: @.***>

CameronBodine commented 4 months ago

Great! I will close.

CameronBodine commented 2 months ago

Hi Dan, I have added the option to clip a sonar recording using an AOI (#110) as specified with a shapefile. You can either generate a shapefile manually in GIS, or create a template shapefile from the sonar recording coverage using a new utility in the utils folder called export_coverage.py (more info in issue #121). If and when you get a chance, I would be interested to hear how it works for you.