Multi-vessel processing and plotting

[sfregosi]

Time to make a plan for including the Lasker!

Discussions of what we want this to look like resulted in the following plan:

• Single map containing both vessels. Will leave map extent as it is and just plot Lasker as it moves into the study area (not worry about sightings in the Eastern Pacific)
• Lasker track will be a yellow/orange/similar color scheme. Sette will remain as is

Code changes need to happen in a few places...

Within run.R:

• @sfregosi will take the lead on this portion
• general approach will be to loop through each vessel individually and do all the necessary processing steps that are already in place, then COMBINE these newly processed outputs into combined effort, sightings, detections dataframes that will then be used to make the map and table
• [x] @sfregosi set up framework to map out what outputs will look like for below
• [x] @sfregosi test manually
• [x] @sfregosi test with automation process

Within plotMap.R:

• @badgerjj will tackle updating the plot_map.R function, modifying the current one in the repository because @sfregosi has made various updates for error checking, legend order, that we need to keep
• general approach will be to use vessel ID columns within all input data.frames to color/label as needed. These include - Cruise (either 2303 or 2401, integer), shipCode (either 'OES' or 'LSK', character), shipName (either 'Sette' or 'Lasker', character), projID (either 'OES2303' or 'LSK2401', character)
• [x] @sfregosi will make example combined datasets to send to @badgerjj
• [x] @badgerjj update function
• [x] @sfregosi test within Run.R

Within makeSummaryTable.R:

• @sfregosi will tackle this
• [x] @sfregosi test with example datasets

[sfregosi]

Sent example .Rda files to @badgerjj (data files aren't uploaded here as we don't necessarily want the raw data on the public repository)

I basically duplicated data from 9/30 and then updated all vessel and cruise info columns to be what they will be for the Lasker and shifted the lat/lon positions by 1 degree (not sure if they will actually still be in EEZ or not).

compiledEffortPoints_twoBoats_combined.rda: loads an epC variable...same as the previous ep variable for 'effort points' but now for both vessels combined newEffortPoints_twoBoats_combined.rda: loads an epNewC variable, like epNew from before, but now combined for both boats. This is ONLY the last day's locations compiledSightings_twoBoats_combined.rda: vsC variable for 'visual sightings combined', for all vessels, all legs compiledDetections_twoBoats_combined.rda: adC variable for 'acoustic detections combined', all legs, both vessels combined

So, for the existing mapping code, before a call for a map of visuals was: plotMap(dir_wd, ep, epNew, vs, shipCode, dataType = 'visual')

Now it would be: plotMap(dir_wd, epC, epNewC, vsC, shipCode, dataType = 'visual') We can drop the shipCode argument because it was just used as a catch for two boats

[sfregosi]

Modified run.R to loop through the two vessels and process any new das or acoustic files for each in turn. The outputs from each iteration of the loop are aggregated into a list, and then that list is 'collapsed' into a combined data.frame that can be used in the mapping and summary table functions.

Still need to build in some catches for missing new das or new acoustic databases, overcome whatever weirdness might happen on the first day of the Lasker, and likely more. But the looping framework is set up.

Also, need to decide about saving these combined datasets. As it is now, it saves each of the intermediate steps on a project ID basis within a vessel/cruise number folder (e.g., OES2303). Could continue to do this for each vessel individually, then have the combined files in the outer 'data' folder, or in a 'combined' folder.

[sfregosi]

More folder structure/saving thoughts:

• Inputs/outputs saved to the 'data' folder are not pushed to the repo (stay local) and are set up with 1 folder per projID (OES2303 and 2401), and generally are vessel specific variables (most won't contain combined data from multiple vessels)
  • Within these are:
    • a 'snapshots' folder of daily .rda outputs
    • a 'gpx' folder of daily .gpx outputs
    • a 'gd_downloads' folder where raw das files and acoustic databases are saved locally
  • The 'compiledSightings_OES2303.rda and .csv' and other 'compiled' .rdas and .csvs are in the outer 'OES2303' folder except for the 'compiledEffortTracks_OES2303.gpx' which is within the 'gpx' folder
  • This is sort of mirrored on Google Drive, but with slightly different structure: each of 'gpx_files', 'processed_data', and 'raw_das_files' folders contain an OES2303 and LSK2401 subfolder where all these different inputs and outputs live
    • 'raw_acoustic_files' just has all databases in one place (they are downloaded by the vessel name and leg within the filename)
• I think these can all stay as they are. All of these outputs are saved and/or uploaded to GD within the processing loop done for each ship individually. The 'gd_downloads' folder will have to stay this way because they DAS files themselves don't have the vessel name in the filename and they would overwrite one another.

Follow up on GPX files... Should these also stay as they are, separated by vessel? Check with EO and all CLs that these will need to remain separate because they are used for individual vessel planning and documenting effort? Or will the Sette crew want to know the Lasker's daily? All CLs will have access to a 'compiledEffortTracks_VES####.gpx' for both vessels so they always could download and add to Coastal Explorer if interested, but this will keep it separate.

---

• Outputs saved to the 'outputs' folder ARE all combined products - maps, tables, and logs
  • Currently, within 'outputs' there are 'map_archive', 'run_logs', and 'table_archive' folders that save daily snapshots of each of the maps and tables and the log of each daily run. Within each of those folders is a folder for each projID and leg (e.g., OES2303_leg1).
  • It does not make sense to save double copies of each daily map with the two different projID_leg strings appended, but doing that would be very straightforward (not much would have to change)

---

Potential changes:

• [x] Move the GD upload location for the 'dailyMap_visuals_CL.pdf' to the outer 'gpx_files' folder as this will soon contain effort tracklines for both vessels. (save location in the local 'outputs' folder is already very central)
• [x] When two vessels are operating, create a new subfolder naming structure with both (e.g., OES2303_leg4_LSK2401_leg1) that would only be created during the overlapping legs and would be used to create subfolders within 'map_archive', 'run_logs', and 'table_archive'.
  • Would we want this to start with the start of OES leg 4, or only after LSK departs?
  • the filename of each daily save would need to be changed as well. Currently 'dailyMap_acoustics_OES2303_leg2_ran2023-08-16.png' but could be changed to 'dailyMap_acoustics_OES2303_leg4_LSK2401_leg1_ran2023-10-30.png' or something like that?

[sfregosi]

Several delayed updates here

• run code is generally operational for processing, saving, and plotting for two vessels. Final step is to test with mapping
• summary table handles two vessels BUT it is still counting sightings the old species (not group) way. Will likely need to manually merge those two files when combining branches. See #39 branch for more info

[sfregosi]

This all seems good to go but going to keep issue open and two-boats branch available until after Lasker is underway and we get first acoustics file from LSK

[sfregosi]

There were still several bugs to fix BUT all working now :) Closing.