Process Saildrone data - Githubissues

kstierhoff commented 1 year ago

Process .raw data collected by Saildrone USVs and verify results by looking at plotCSV_Saildrone: https://drive.google.com/file/d/1S3L08AmHbmlW-FGg_fsOO1dVdleunh5u/view?usp=share_link

kstierhoff commented 1 year ago

Please check the following:

[x] loop nearshore on 152
[x] partial line for 146
[x] gap in 130 (likely real, just checking)
[x] overlap on 126
[x] partial line on 160

alicebeittel commented 1 year ago

I re exported: 142-2 Steve re exported: 126-1 138-1 146-2 152-1 160-1 160-2

Gap in 130 is real. No SD data between the two SD. All except 138-1 have been reprocessed in nasc.r

Tasks Remaining: 138-1 - seabed included at far right of transect -- issue with integration stop line? 146-2 - check gps files in EV + re run nasc.r

@kstierhoff updated!

kstierhoff commented 1 year ago

All done.

alicebeittel commented 1 year ago

Reopen!

kstierhoff commented 1 year ago

Not done.

jrenfree commented 1 year ago

@alicebeittel @kstierhoff What is the actual status of this? Should we move this back into the "In Progress" column or is this actually done?

alicebeittel commented 1 year ago

@jrenfree @stevesessions Re-ran 138-1 and 146-2 through extract_CPS_nasc.r.

SD processing complete, but leave "in progress". There are some lines that I think need a second opinion on, marked in processing log. Need to go though google doc and spreadsheet and look at what to bring to group.

jrenfree commented 1 year ago

@alicebeittel is this supposed to be in the "Done" column? Sounded like there was more work that needed to be done.

alicebeittel commented 1 year ago

I am compiling the transects that need review for nasc.r processing into a google doc. I'll share the doc tomorrow with everyone so folks can give input. They are areas where I'm not sure whether to include or exclude the backscatter. Leave as "in progress" for the moment. Slow in turning this around due to the echoview course this week!

jrenfree commented 1 year ago

OK, I'll move it back to the In Progress column then

alicebeittel commented 1 year ago

I've compiled some images from EV exported echograms and EV graphs for areas along SD transects that I think warrant the group to look at and provide input. See the section in the below google doc called "For Group Review". Can folks check out the doc below and comment below each image what you think is going on and I can review our discussion and then make any edits to the nasc.r graphs as needed. After this I'll feel better about saying the SD nasc.r processing is done. I've put a standing open hour on the calendar for tomorrow at 11-12 for anyone who is available to go over some of the example over video chat!

@ddemer @kstierhoff @jrenfree @dmurfin @stevesessions @ScottMau

jrenfree commented 1 year ago

@alicebeittel can you let me know when you finish processing the Saildrone data? I'd like to copy that data onto an external hard drive so that I can do some extra work on it using the remote processing machine.

Thanks!

alicebeittel commented 1 year ago

Group revisited tricky lines. Documented changes in EV Processing Google Excel sheet. Saildrone processing in nasc.r done. Don't make any changes to SD data on AST 4 at this time - @jrenfree will be copying data for reprocessing

jrenfree commented 1 year ago

The main task has been completed, so going ahead and closing this issue.

jrenfree commented 1 year ago

Re-opening this because:

[x] The CSV file for SD1076_130-1 appears to be incomplete (should have 388 Intervals but CSV file only contains 193)
[x] It appears that the EV file for Transect SD1077_160-2 contains .raw files from SD1076
[x] There is so CSV file for Transect SD1077_150-2 despite there being an EV file (no data for this transect)

@stevesessions

stevesessions commented 1 year ago

Dang.

I can address that tomorrow

Thanks for pointing that out. Much appreciated the QC

On Sat, Nov 19, 2022 at 9:34 PM Josiah Renfree @.***> wrote:

Re-opening this as it appears that the EV file for Transect SD1077_160-2 contains .raw files from SD1076.

@stevesessions https://github.com/stevesessions

— Reply to this email directly, view it on GitHub https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1321043971, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4D3CKET3K56GYW77KPOO7LWJGZ5XANCNFSM6AAAAAAR32K5E4 . You are receiving this because you were mentioned.Message ID: @.***>

-- Steve Sessions Equipment Specialist Advanced Survey Technologies Program NOAA / NMFS / Southwest Fisheries Science Center 8901 La Jolla Shores Drive La Jolla, CA 92037 Office: (858) 546-7097 Technology Lab: (858) 334-2857

jrenfree commented 1 year ago

Great, thanks. I'm mostly concerned with Transect SD1077_160-2, as it'll probably be easier for you to determine what happened. The other two transects I can easily re-export then run through NASC(R)

stevesessions commented 1 year ago

Josiah, Copy that. I will grab a dongle and address it first thing tomorrow.

Steve

On Mon, Nov 21, 2022 at 7:38 AM Josiah Renfree @.***> wrote:

Great, thanks. I'm mostly concerned with Transect SD1077_160-2, as it'll probably be easier for you to determine what happened. The other two transects I can easily re-export then run through NASC(R)

— Reply to this email directly, view it on GitHub https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322250556, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4D3CKDUAWL2GSDPXTXTBFLWJOJPLANCNFSM6AAAAAAR32K5E4 . You are receiving this because you were mentioned.Message ID: @.***>

-- Steve Sessions Equipment Specialist Advanced Survey Technologies Program NOAA / NMFS / Southwest Fisheries Science Center 8901 La Jolla Shores Drive La Jolla, CA 92037 Office: (858) 546-7097 Technology Lab: (858) 334-2857

alicebeittel commented 1 year ago

I'm at the office today + have a dongle and can take a look at SD1077_160-2!

LTJG Alice Beittel, NOAA Corps Advanced Survey Technology Officer NOAA Southwest Fisheries Science Center 8901 La Jolla Shores Drive San Diego, CA 92037 Google Voice: (541) 283-3756 Cell: (510) 847-2048

On Mon, Nov 21, 2022 at 7:44 AM stevesessions @.***> wrote:

Josiah, Copy that. I will grab a dongle and address it first thing tomorrow.

Steve

On Mon, Nov 21, 2022 at 7:38 AM Josiah Renfree @.***> wrote:

Great, thanks. I'm mostly concerned with Transect SD1077_160-2, as it'll probably be easier for you to determine what happened. The other two transects I can easily re-export then run through NASC(R)

— Reply to this email directly, view it on GitHub < https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322250556 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/A4D3CKDUAWL2GSDPXTXTBFLWJOJPLANCNFSM6AAAAAAR32K5E4

. You are receiving this because you were mentioned.Message ID: @.***>

-- Steve Sessions Equipment Specialist Advanced Survey Technologies Program NOAA / NMFS / Southwest Fisheries Science Center 8901 La Jolla Shores Drive La Jolla, CA 92037 Office: (858) 546-7097 Technology Lab: (858) 334-2857

— Reply to this email directly, view it on GitHub https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322259230, or unsubscribe https://github.com/notifications/unsubscribe-auth/A236EGRJA34G435JDBBKDT3WJOKEDANCNFSM6AAAAAAR32K5E4 . You are receiving this because you were mentioned.Message ID: @.***>

stevesessions commented 1 year ago

Alice, Copy that. Thanks very much for taking a look.

Steve

On Mon, Nov 21, 2022 at 8:40 AM Alice Beittel @.***> wrote:

I'm at the office today + have a dongle and can take a look at SD1077_160-2!

LTJG Alice Beittel, NOAA Corps Advanced Survey Technology Officer NOAA Southwest Fisheries Science Center 8901 La Jolla Shores Drive San Diego, CA 92037 Google Voice: (541) 283-3756 Cell: (510) 847-2048

On Mon, Nov 21, 2022 at 7:44 AM stevesessions @.***> wrote:

Josiah, Copy that. I will grab a dongle and address it first thing tomorrow.

Steve

On Mon, Nov 21, 2022 at 7:38 AM Josiah Renfree @.***> wrote:

Great, thanks. I'm mostly concerned with Transect SD1077_160-2, as it'll probably be easier for you to determine what happened. The other two transects I can easily re-export then run through NASC(R)

— Reply to this email directly, view it on GitHub <

https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322250556

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/A4D3CKDUAWL2GSDPXTXTBFLWJOJPLANCNFSM6AAAAAAR32K5E4

. You are receiving this because you were mentioned.Message ID: @.***>

-- Steve Sessions Equipment Specialist Advanced Survey Technologies Program NOAA / NMFS / Southwest Fisheries Science Center 8901 La Jolla Shores Drive La Jolla, CA 92037 Office: (858) 546-7097 Technology Lab: (858) 334-2857

— Reply to this email directly, view it on GitHub < https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322259230 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/A236EGRJA34G435JDBBKDT3WJOKEDANCNFSM6AAAAAAR32K5E4

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322346414, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4D3CKGZ6IECPFSWNIK23TLWJOQYRANCNFSM6AAAAAAR32K5E4 . You are receiving this because you were mentioned.Message ID: @.***>

-- Steve Sessions Equipment Specialist Advanced Survey Technologies Program NOAA / NMFS / Southwest Fisheries Science Center 8901 La Jolla Shores Drive La Jolla, CA 92037 Office: (858) 546-7097 Technology Lab: (858) 334-2857

alicebeittel commented 1 year ago

Fixed 2207SD1077_160-2_CPS in EV 12. Did not run the export_2207SD_all script since it automatically opens up my EV 13. Let me know if there is anything else!

LTJG Alice Beittel, NOAA Corps Advanced Survey Technology Officer NOAA Southwest Fisheries Science Center 8901 La Jolla Shores Drive San Diego, CA 92037 Google Voice: (541) 283-3756 Cell: (510) 847-2048

On Mon, Nov 21, 2022 at 8:45 AM stevesessions @.***> wrote:

Alice, Copy that. Thanks very much for taking a look.

Steve

On Mon, Nov 21, 2022 at 8:40 AM Alice Beittel @.***> wrote:

I'm at the office today + have a dongle and can take a look at SD1077_160-2!

LTJG Alice Beittel, NOAA Corps Advanced Survey Technology Officer NOAA Southwest Fisheries Science Center 8901 La Jolla Shores Drive San Diego, CA 92037 Google Voice: (541) 283-3756 Cell: (510) 847-2048

On Mon, Nov 21, 2022 at 7:44 AM stevesessions @.***> wrote:

Josiah, Copy that. I will grab a dongle and address it first thing tomorrow.

Steve

On Mon, Nov 21, 2022 at 7:38 AM Josiah Renfree @.***> wrote:

Great, thanks. I'm mostly concerned with Transect SD1077_160-2, as it'll probably be easier for you to determine what happened. The other two transects I can easily re-export then run through NASC(R)

— Reply to this email directly, view it on GitHub <

https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322250556

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/A4D3CKDUAWL2GSDPXTXTBFLWJOJPLANCNFSM6AAAAAAR32K5E4

. You are receiving this because you were mentioned.Message ID: @.***>

-- Steve Sessions Equipment Specialist Advanced Survey Technologies Program NOAA / NMFS / Southwest Fisheries Science Center 8901 La Jolla Shores Drive La Jolla, CA 92037 Office: (858) 546-7097 Technology Lab: (858) 334-2857

— Reply to this email directly, view it on GitHub <

https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322259230

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/A236EGRJA34G435JDBBKDT3WJOKEDANCNFSM6AAAAAAR32K5E4

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322346414 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/A4D3CKGZ6IECPFSWNIK23TLWJOQYRANCNFSM6AAAAAAR32K5E4

. You are receiving this because you were mentioned.Message ID: @.***>

-- Steve Sessions Equipment Specialist Advanced Survey Technologies Program NOAA / NMFS / Southwest Fisheries Science Center 8901 La Jolla Shores Drive La Jolla, CA 92037 Office: (858) 546-7097 Technology Lab: (858) 334-2857

— Reply to this email directly, view it on GitHub https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322351496, or unsubscribe https://github.com/notifications/unsubscribe-auth/A236EGQ4ASXKCFUJPM5RL2TWJORJPANCNFSM6AAAAAAR32K5E4 . You are receiving this because you were mentioned.Message ID: @.***>

stevesessions commented 1 year ago

Thanks Alice.

I can export tomorrow morning with Echoview 12

On Mon, Nov 21, 2022 at 11:06 AM Alice Beittel @.***> wrote:

Fixed 2207SD1077_160-2_CPS in EV 12. Did not run the export_2207SD_all script since it automatically opens up my EV 13. Let me know if there is anything else!

LTJG Alice Beittel, NOAA Corps Advanced Survey Technology Officer NOAA Southwest Fisheries Science Center 8901 La Jolla Shores Drive San Diego, CA 92037 Google Voice: (541) 283-3756 Cell: (510) 847-2048

On Mon, Nov 21, 2022 at 8:45 AM stevesessions @.***> wrote:

Alice, Copy that. Thanks very much for taking a look.

Steve

On Mon, Nov 21, 2022 at 8:40 AM Alice Beittel @.***> wrote:

I'm at the office today + have a dongle and can take a look at SD1077_160-2!

LTJG Alice Beittel, NOAA Corps Advanced Survey Technology Officer NOAA Southwest Fisheries Science Center 8901 La Jolla Shores Drive San Diego, CA 92037 Google Voice: (541) 283-3756 Cell: (510) 847-2048

On Mon, Nov 21, 2022 at 7:44 AM stevesessions @.***> wrote:

Josiah, Copy that. I will grab a dongle and address it first thing tomorrow.

Steve

On Mon, Nov 21, 2022 at 7:38 AM Josiah Renfree @.***> wrote:

Great, thanks. I'm mostly concerned with Transect SD1077_160-2, as it'll probably be easier for you to determine what happened. The other two transects I can easily re-export then run through NASC(R)

— Reply to this email directly, view it on GitHub <

https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322250556

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/A4D3CKDUAWL2GSDPXTXTBFLWJOJPLANCNFSM6AAAAAAR32K5E4

. You are receiving this because you were mentioned.Message ID: @.***>

-- Steve Sessions Equipment Specialist Advanced Survey Technologies Program NOAA / NMFS / Southwest Fisheries Science Center 8901 La Jolla Shores Drive La Jolla, CA 92037 Office: (858) 546-7097 Technology Lab: (858) 334-2857

— Reply to this email directly, view it on GitHub <

https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322259230

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/A236EGRJA34G435JDBBKDT3WJOKEDANCNFSM6AAAAAAR32K5E4

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub <

https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322346414

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/A4D3CKGZ6IECPFSWNIK23TLWJOQYRANCNFSM6AAAAAAR32K5E4

. You are receiving this because you were mentioned.Message ID: @.***>

-- Steve Sessions Equipment Specialist Advanced Survey Technologies Program NOAA / NMFS / Southwest Fisheries Science Center 8901 La Jolla Shores Drive La Jolla, CA 92037 Office: (858) 546-7097 Technology Lab: (858) 334-2857

— Reply to this email directly, view it on GitHub < https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322351496 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/A236EGQ4ASXKCFUJPM5RL2TWJORJPANCNFSM6AAAAAAR32K5E4

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322519137, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4D3CKEX5ORHEINDWP7UWWDWJPBZHANCNFSM6AAAAAAR32K5E4 . You are receiving this because you were mentioned.Message ID: @.***>

-- Steve Sessions Equipment Specialist Advanced Survey Technologies Program NOAA / NMFS / Southwest Fisheries Science Center 8901 La Jolla Shores Drive La Jolla, CA 92037 Office: (858) 546-7097 Technology Lab: (858) 334-2857

jrenfree commented 1 year ago

Hey Alice could you let me know where the re-processed 2207SD1077_160-2_CPS files are. Are they on AST4?

alicebeittel commented 1 year ago

Yep - the EV file is on AST 4 but still needs to be exported. Meant to export it yesterday but totally slipped my mind after shifting focus to LBC data!

On Mon, Nov 21, 2022 at 17:37 Josiah Renfree @.***> wrote:

Hey Alice could you let me know where the re-processed 2207SD1077_160-2_CPS files are. Are they on AST4?

— Reply to this email directly, view it on GitHub https://github.com/nmfs-swfsc-ast/ast-tasks/issues/1#issuecomment-1322886527, or unsubscribe https://github.com/notifications/unsubscribe-auth/A236EGQKYPCZEQN3NUHHMP3WJQPT5ANCNFSM6AAAAAAR32K5E4 . You are receiving this because you were mentioned.Message ID: @.***>

-- LTJG Alice Beittel, NOAA Corps Advanced Survey Technology Officer NOAA Southwest Fisheries Science Center 8901 La Jolla Shores Drive San Diego, CA 92037 Google Voice: (541) 283-3756 Cell: (510) 847-2048

kstierhoff commented 1 year ago

@stevesessions has re-exported this file.

nmfs-swfsc-ast / ast-tasks

Process Saildrone data #1