afsc-gap-products / data-requests

An empty repo for tracking requests for GAP data
0 stars 0 forks source link

Incorrect Gear Depth of 0 for Vessel 148, Cruise 202201, Haul 191 #66

Open Ned-Laman-NOAA opened 1 year ago

Ned-Laman-NOAA commented 1 year ago

Gear depth at this locus in RACEBase is 0 (zero). It looks like the BT cast was bad and no data were collected during the On-Off Bottom period. Furthermore, the Null that would've resulted for Gear Depth (and probably Gear Temperature and maybe Surface Temperature) was translated to a 0 in RACEBase.

The fix for this is to extract values of temperature and depth from the companion CTD cast from this haul. Insert those new values with appropriate method codes into RACE_DATA and refresh the materialized view of RACEBase.HAUL.

NancyRoberson commented 1 year ago

Hi Ned and Megsie,

I found where the CTD data reside G:\RACE_CTD\data\2022\AI\v148_8103\202201_148_L3\2022_07_31 but have no idea how to extract the temperature and depth. I am looping in Sean for his recommendations how to do this part of the fix.

MargaretSiple-NOAA commented 1 year ago

Hi Nancy! I know how to do this using the functions Sean put in the GAPSurvey package, but I may not get to it right away. I'll try to do it this week but will request backup if I don't get to it before we get on the boat.

NancyRoberson commented 1 year ago

Thanks Megsie and that sounds like a good plan. I can work on the RACE_DATA editing part of the fix unless you want to do it? :)

MargaretSiple-NOAA commented 1 year ago

Noooo you should do that part! 🚑 I will send you a message when I have the numbers and I'll make sure you get what you need. Thank you!

sean-rohan-NOAA commented 1 year ago

I can handle the conversion to .BTD and .BTH because the function in GAPsurvey currently requires an installation SBE Data Processing. I wrote a function for the gapctd packages that converts .hex files without using the software but I used dependencies that cannot be used in the GAPsurvey package.

BTD and BTH files for haul 191 (.zip)

Conversion code

# Requires an installation of SBE Data Processing on your computer
install.packages("remotes")
library(remotes)
remotes::install_github("afsc-gap-products/GAPsurvey") # GAPsurvey_2023.04.01

GAPsurvey::convert_ctd_btd(VESSEL = 148, CRUISE = 202201, HAUL = 191, SERIAL_NUMBER = 8103,
                           path_in = "G:\\RACE_CTD\\data\\2022\\AI\\v148_8103\\202201_148_L3\\2022_07_31\\SBE19plus_01908103_2022_07_31_0003.hex",
                           path_xmlcon = "G:\\RACE_CTD\\xmlcon\\Calibration\\2022\\xmlcon config\\All 2022 Calibration Files and Docs from Sea-Bird\\CON & INI Files\\SBE19plusV2_8103.xmlcon" # Path to the 2022 configuration file for CTD w/ serial number ending in 8103
Ned-Laman-NOAA commented 1 year ago

Thanks for the BT files Sean. I've replaced the original Haul0191.BT* files in G:\ALEUTIAN\AI 2022\Data\Ocean Explorer\Leg 3\HAUL with the ones for the CTD that Sean extracted for us. The original files can be found here G:\ALEUTIAN\AI 2022\Data\Ocean Explorer\Leg 3\HAUL\Haul0191 Original BT files. I'm going to need a brain refresh for this since we're a way out from data editing season, but I think (correct me if I'm wrong) we need to run these new BT files through Calypso and Wheelhouse to get the calculated/selected fields (gear_temperature, gear_depth, bottom_depth, and surface_temperature). Then we need to send the data streams to the @sea data repository which will result in the old streams being overwritten by the new in RACE_DATA. Then we need to edit method codes in GIDES to reflect that the data come from the CTD and not he SBE39.

NancyRoberson commented 1 year ago

For AI 2022, Vessel 148, Haul 191 there already exists Bottom Temp, Surface Temp, Net Height, and Bottom Depth in RACEBASE Haul. Only the Gear Depth is NULL.

I can calculate Gear Depth by subtracting Net Height from Bottom Depth and that is much simpler than redoing everything in Wheelhouse and Calypso.

Is that an okay approach?

MargaretSiple-NOAA commented 1 year ago

@Ned-Laman-NOAA I am checking back in on this because I directed someone else to our data request page and realized this one was in progress! It sounds like Nancy's solution will work, so if you're now in data editing mode Ned, let us know if that is the way to go.

Ned-Laman-NOAA commented 1 year ago

I stand by my note from May 10 indicating that what still needs to happen here is that the recovered BT data streams that Sean provided need to be hand walked through Wheelhouse in the shop and then processed through Calypso to get the calculated fields from our standard data grinding machinery. Then all of Haul191 needs to be extracted from Wheelhouse into 7zip form to be ready for upload through the @sea FTP portal. Before uploading the data, the whole cruise needs to be moved from Final to Edit in GIDES. Now uploading haul 191 (every component of it) will overwrite haul 191 in the Edit tables in GIDES. Once there, edits will need to be made to method codes to reflect that the BT data did not come from the SBE. Then edit checks need to be run and addressed, then data need to be refinalized, and migrated back to the Final tables. I confirmed this morning that there is no BT data stream recorded for Haul 191 in the Final tables in RACE_DATA and do not recommend simply editing the summary values at the haul level. We have the recovered CTD data and can populated the database so that the derived valuables can be reproduced from the observation-level data; this is what should be done.

NancyRoberson commented 1 year ago

Thanks @Ned-Laman-NOAA , This cruise is currently in the Edit tables and available for editing.