search

nmfs-swfsc-ast / ast-tasks

0 stars 0 forks source link

Transfer data to network drives #38

Open jrenfree opened 1 year ago

jrenfree commented 1 year ago

This issue is to detail plans or ask questions regarding data transfers from the Synology or external hard drives to the AST network drives. Once data is on the AST network and backed up to tape, they can be removed from the Synology or external hard drives in order to free up space for the surveys.

alicebeittel commented 8 months ago

There is currently a folder called "20230708_SHIMADA_SummerCPS" on AST4. Should the date be changed to 20231010 since that is when we the project instructions began on Shimada? I went to create the Shimada folder on NAS1 (mirroring AST4) to separate the echosounder data from Shimada out of the 2307RL folder and saw that the date was did not reflect the project start date on AST4.

jrenfree commented 8 months ago

@kstierhoff Do you have a preference on this? Don't think it really matters in the long run.

kstierhoff commented 8 months ago

I don't have a strong preference. I don't think it matters. But I think I've typically tried to have the date match the project start dates.

alicebeittel commented 8 months ago

The raw files have 2310SH not 2307SH, so changing it would match the file names. Also the raw EK80 files from Leg 4.2 are not named correctly, they are missing the 2310SH_EK80_CW or FM portion of the file name. Is there a way to tell which files are CW and which are FM? I can rename them in bulk, but not sure about the CW and FM.

jrenfree commented 8 months ago

So when you say match the project start dates you mean all vessels would have the same date?

FYI, DO NOT change the survey directory names on AST4 (or any of the network drives). That should be coordinated with Rich to occur just before a full system backup is performed.

alicebeittel commented 8 months ago

Correct. I am not changing anything on AST 4, only changing on NAS1. I'm confused on your question, so I'll just stop by your office.

kstierhoff commented 8 months ago

I mean match the project start date for the associated vessel, so e.g., the dates for RL would/could be different than SH, LBC, and LM.

jrenfree commented 8 months ago

Would the folder name match the intended start date for each vessel, or the start date that was ultimately achieved?

alicebeittel commented 8 months ago

Posting an update here: I removed the Shimada raw EK80 and ME70 data from the 2307RL folder on NAS1 and put it into the 20231010_SHIMADA_SummerCPS folder on NAS1. I have not moved any of the processed data from Shimada to the new folder as Josiah will do this.

kstierhoff commented 8 months ago

These directories typically get created early in the process, before project instructions are completed in most cases, so it's usually named long before any loss of sea days occur. So no need to adjust based on date ranges of actual data, IMO.