search

npolar / marine-db

https://doi.org/10.21334/marine-db
0 stars 0 forks source link

Create supplemental events for IOPAN legacy protists #62

Open cnrdh opened 2 years ago

cnrdh commented 2 years ago

Some occurrences fails to match with events via fieldNumber and expedition. Sometimes due to ABC suffixing.

cnrdh commented 2 years ago 

"bottomDepth"   "decimalLatitude"   "decimalLongitude"  "eventDate" "eventRemarks"  "expedition"    "fieldNumber"   "locationID"    "maximumDepthInMeters"  "minimumDepthInMeters"  "modified"  "type"  "year"  "gear"  "sampletype"
187 80.605  22.1166 "2010-08-19T06:15:00Z"  "ICE10-16-7; Chl a max" "ICE2010"   "ICE10-158" "R4"    38  38  "2020-06-11T07:25:16.805Z"  "MaterialSample"    2010        
187 80.605  22.1166 "2010-08-19T06:15:00Z"  "ICE10-16-7"    "ICE2010"   "ICE10-155" "R4"    25  25  "2020-06-11T07:25:16.803Z"  "MaterialSample"    2010        
187 80.605  22.1166 "2010-08-19T06:15:00Z"  "ICE10-16-7"    "ICE2010"   "ICE10-156" "R4"    50  50  "2020-06-11T07:25:16.804Z"  "MaterialSample"    2010        
187 80.605  22.1166 "2010-08-19T06:15:00Z"  "ICE10-16-7"    "ICE2010"   "ICE10-157" "R4"    100 100 "2020-06-11T07:25:16.804Z"  "MaterialSample"    2010        
187 80.605  22.1166 "2010-08-19T06:15:00Z"  "ICE10-16-7"    "ICE2010"   "ICE10-157" "R4"    100 100 "2020-06-11T07:25:16.805Z"  "MaterialSample"    2010        
187 80.605  22.1166 "2010-08-19T06:15:00Z"  "ICE10-16-7"    "ICE2010"   "ICE10-159" "R4"    0   0   "2020-06-11T07:25:16.805Z"  "MaterialSample"    2010        
644 81.4673 22.5182 "2010-08-22T07:34:00Z"  "ICE10-16-13"   "ICE2010"   "ICE10-384" "ICE10-16-13"   100 100 "2020-06-11T07:25:16.842Z"  "MaterialSample"    2010        
644 81.4673 22.5182 "2010-08-22T07:34:00Z"  "ICE10-16-13"   "ICE2010"   "ICE10-383" "ICE10-16-13"   50  50  "2020-06-11T07:25:16.842Z"  "MaterialSample"    2010        
644 81.4673 22.5182 "2010-08-22T07:34:00Z"  "ICE10-16-13; Chla max" "ICE2010"   "ICE10-382" "ICE10-16-13"   25  25  "2020-06-11T07:25:16.842Z"  "MaterialSample"    2010        
644 81.4673 22.5182 "2010-08-22T07:34:00Z"  "ICE10-16-13"   "ICE2010"   "ICE10-381" "ICE10-16-13"   10  10  "2020-06-11T07:25:16.841Z"  "MaterialSample"    2010        
644 81.4673 22.5182 "2010-08-22T07:34:00Z"  "ICE10-16-13"   "ICE2010"   "ICE10-380" "ICE10-16-13"   5   5   "2020-06-11T07:25:16.841Z"  "MaterialSample"    2010        
644 81.4673 22.5182 "2010-08-22T07:34:00Z"  "ICE10-16-13"   "ICE2010"   "ICE10-379" "ICE10-16-13"   0   0   "2020-06-11T07:25:16.840Z"  "MaterialSample"    2010        
65  77.8317 16.4865 "2016-05-18T04:00:00Z"      "ARCEx2016" "PHT-HN1"   "vMf1"  20  0       "MaterialSample"    2016    "Handnet"   "Phytoplankton taxonomy"
    78.9733 12.4623 "2017-07-28T10:30:00Z"      "GlacierFront2017"  "HAN-Cc1"   "Cc1"   20              2017    "Handnet"   
    78.9773 12.4423 "2017-07-28T09:25:00Z"      "GlacierFront2017"  "HAN-Cc2"   "Cc2"   20              2017    "Handnet"   
    78.9809 12.4532 "2017-07-28T07:13:00Z"      "GlacierFront2017"  "HAN-Cc3"   "Cc3"   20              2017    "Handnet"   
    78.9899 12.4999 "2017-07-28T13:45:00Z"      "GlacierFront2017"  "HAN-CpN3"  "CpN3"  20              2017    "Handnet"   
    78.9785 12.489  "2017-07-28T17:40:00Z"      "GlacierFront2017"  "HAN-CpS1"  "CpS1"  20              2017    "Handnet"   
    78.931  12.382  "2017-07-29T00:45:00Z"      "GlacierFront2017"  "HAN-Kb6"   "Kb6"   20              2017    "Handnet"   
    78.8815 12.4987 "2017-07-26T16:50:00Z"      "GlacierFront2017"  "HAN-KpM3"  "KpM3"  20              2017    "Handnet"   
    78.8778 12.5297 "2017-07-26T09:00:00Z"      "GlacierFront2017"  "HAN-KpM4"  "KpM4"  20              2017    "Handnet"   
    78.8737 12.5518 "2017-07-25T15:20:00Z"      "GlacierFront2017"  "HAN-KpM5"  "KpM5"  20              2017    "Handnet"   
    78.894  12.4652 "2017-07-27T08:00:00Z"      "GlacierFront2017"  "HAN-KpN2"  "KpN2"  20              2017    "Handnet"   
    78.8912 12.5137 "2017-07-26T19:00:00Z"      "GlacierFront2017"  "HAN-KpN3"  "KpN3"  20              2017    "Handnet"   
    78.8863 12.5656 "2017-07-25T20:40:00Z"      "GlacierFront2017"  "HAN-KpN5"  "KpN5"  20              2017    "Handnet"   
    78.8758 12.4994 "2017-07-26T13:20:00Z"      "GlacierFront2017"  "HAN-KpS3"  "KpS3"  20              2017    "Handnet"   
    78.8703 12.5178 "2017-07-26T12:00:00Z"      "GlacierFront2017"  "HAN-KpS4"  "KpS4"  20              2017    "Handnet"   
    78.8655 12.5542 "2017-07-25T08:53:00Z"      "GlacierFront2017"  "HAN-KpS5"  "KpS5"  20              2017    "Handnet"   
                    "GlacierFront2017"  "__HAN-KC4_Kb5__"   "KC4_Kb5"                       "Handnet"
cnrdh commented 2 years ago

List of non-matching samples:


["ICE2010","ICE10-152","R4",0,null]
["ICE2010","ICE10-253","R6b",5,5]
["ICE2010","ICE10-undefined","16-13",20,20]
["ICE2010","ICE10-undefined","ICE-22",20,20]
["ICE2010","ICE10-undefined","R10b",20,20]
["ICE2010","ICE10-undefined","R7b",20,20]
["ICE2010","ICE10-undefined","R8b",20,20]
["ICE2010","ICE10-undefined","R9b",20,20]
["ICE2012","Agneta","Underice",2,2]
["ICE2012","Core2.1","1mbelowice",null,null]
["ICE2012","Core2.1","undericehole",null,null]
["ICE2012","Divehole","5mdepth",null,null]
["ICE2012","ICE12-760","Floe1",47,47]
["ICE2012","ICE12-822","Floe1",32,32]
["ICE2012","ICE12-996a","Floe1",36,36]
["ICE2012","Pond","handnet20um",null,null]
["ICE2012","Ridge","handnet20um",null,null]
["PolarNight2012","Rijpfjorden_0m_2012-01-12",null,0,null]
["PolarNight2012","Rijpfjorden_15m_2012-01-12",null,15,15]
["PolarNight2012","Rijpfjorden_35m_2012-01-12",null,35,35]
["PolarNight2012","Rijpfjorden_75m_2012-01-12",null,75,75]
["PolarNight2012","Rijpfjorden_m_2012-01-12",null,null,null]
["PolarNight2012","Sofiadypet_0m_2012-01-12",null,0,null]
["PolarNight2012","Sofiadypet_150m_2012-01-14",null,150,150]
["PolarNight2012","Sofiadypet_15m_2012-01-14",null,15,15]
["PolarNight2012","Sofiadypet_5m_2012-01-14",null,5,5]
["PolarNight2012","Sofiadypet_75m_2012-01-14",null,75,75]
["MOSJ2015","PHT-0?",null,null,null]
["N-ICE2015","PHT-144","inlogdoesnotexist",null,null]
["N-ICE2015","PHT-202","Floe3","in log does not exist",null]
["N-ICE2015","IAT-018",null,null,null]
["N-ICE2015","IAT-021",null,null,null]
["N-ICE2015","IAT-022",null,null,null]
["N-ICE2015","IAT-023",null,null,null]
["N-ICE2015","IAT-024",null,null,null]
["N-ICE2015","IAT-025",null,null,null]
["N-ICE2015","IAT-026",null,null,null]
["N-ICE2015","IAT-027",null,null,null]
["N-ICE2015","IAT-028",null,null,null]
["N-ICE2015","IAT-029",null,null,null]
["N-ICE2015","IAT-030",null,null,null]
["N-ICE2015","IAT-041",null,null,null]
["N-ICE2015","IAT-072",null,null,null]
["N-ICE2015","IAT-073",null,null,null]
["N-ICE2015","IAT-099",null,null,null]
["N-ICE2015","IAT-100",null,null,null]
["N-ICE2015","IAT-280",null,null,null]
["MOSJ2016","MIT-015","KB1_KB2",25,25]
["MOSJ2016","HAN-018","R6",20,20]
["GlacierFront2017","PHT-031","CpS1",1,1]
["GlacierFront2017","HAN-KC4_Kb5","KC4_Kb5",20,20]
AnetteWold commented 2 years ago

Info about missing sample log records for ICE 2010: ["ICE2010","ICE10-152","R4",0,null] could be ICE10-159 if this doesn't already have data attached to it ["ICE2010","ICE10-253","R6b",5,5] add samplelog info: [ICE2010, ICE10-253, R6b, 81.1379, 22.7625, 215, 20.08.2010, 02:30, Niskin bottle, 5.5, 5.5, Phytoplankton taxonomy, Philipp Assmy] ["ICE2010","ICE10-undefined","16-13",20,20] add samplelog info: [ICE2010, blancID, ICE10-16-13, 81.4673, 22.5182, 644, 22.08.2010, 07:34, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy] ["ICE2010","ICE10-undefined","ICE-22",20,20] add samplelog info: [ICE2010, blanc, Ice22, 81.5115, 30.1349 860, 29.08.2010, 08:25, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy] ["ICE2010","ICE10-undefined","R10b",20,20] add samplelog info: [ICE2010, blanc, R10b, 81.7168, 23.1800, 2798, 23.08.2010, 13:00, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy] ["ICE2010","ICE10-undefined","R7b",20,20] add samplelog info: [ICE2010, blanc, R7b, 81.3044, 21.9615, 368, 21.08.2010, 02:40, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy] ["ICE2010","ICE10-undefined","R8b",20,20] add samplelog info: [ICE2010, blanc, R8b, 81.3892, 22.3506, 508, 21.08.2010, 14:15, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy] add samplelog info: [ICE2010, blanc, R9b, 81.5532, 22.6899, 889, 22.08.2010 16:10, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy]

Si ifra hvis du ønsker tilbakemelding på annet format. Jeg sender også oppdattert samplelog2010 der missing records er lagt til i rad [882-888] på epost

cnrdh commented 2 years ago

Ok, fint. Det er ingen data med ICE10-159, se også #51 Antall linjer som går tapt per nu:


$ cat data/input/iopan/protist-biodiversity/log/error-expeditions-occurrences-not-found.txt 

1183
    461 [2010,"ICE2010"]
    219 [2012,"ICE2012"]
     40 [2012,"PolarNight2012"]
      3 [2015,"MOSJ2015"]
    330 [2015,"N-ICE2015"]
     20 [2016,"ARCEx2016"]
     39 [2016,"MOSJ2016"]
     71 [2017,"GlacierFront2017"]
AnetteWold commented 2 years ago

Info missing records ICE 2012: ["ICE2012","Core2.1","1mbelowice",null,null]: No records of station or date. No record of Core2.1 in samplelog ["ICE2012","Core2.1","undericehole",null,null]: No records of station or date. No record of Core2.1 in samplelog ["ICE2012","Divehole","5mdepth",null,null]: No records of station or date. One Station called Dive Site 1 [ Dive site 1, 82.3573, 20.6642, 3751, 28.07.2012, 11:30] ["ICE2012","ICE12-760","Floe1",47,47]: record found [ICE2012, ICE12-760, Floe1, 82.3483, 21.2582, 3746, 28.07.2012, 06:54, Ship CTD, 47, Chl a max, Phytoplankton taxonomy, Philipp Assmy] ["ICE2012","ICE12-822","Floe1",32,32]: record found [ICE2012, ICE12-822, Floe1, 82.3627, 21.5692, 3786, 30.07.2012, 08:00, Niskin bottle, 32, Chl a max, Phytoplankton taxonomy, Philipp Assmy] ["ICE2012","ICE12-996a","Floe1",36,36] record found [ICE2012, ICE12-996a, Floe1, 82.0522, 21.9165, 3603, 02.08.2012, 06:30, Niskin bottle, 36, Phytoplankton taxonomy,Philipp Assmy] ["ICE2012","Pond","handnet20um",null,null] No records of station or date. Could assume that it is from Floe1 (check w/ Philipp) ["ICE2012","Ridge","handnet20um",null,null] Could assume that it is from Floe1 (check w/ Philipp)

AnetteWold commented 2 years ago

Info missing records Polar Night Cruise 2021: ["PolarNight2012","Rijpfjorden_0m_2012-01-12",null,0,null] ["PolarNight2012","Rijpfjorden_15m_2012-01-12",null,15,15] ["PolarNight2012","Rijpfjorden_35m_2012-01-12",null,35,35] ["PolarNight2012","Rijpfjorden_75m_2012-01-12",null,75,75] Only one station in Rjipjorden and one CTD with water samples so all samples can have the following info with the different sampling depths given (0, 15, 35, 75) [PolarNight2012, R3, 80.3098, 22.2599, 274, 12.01.2012, 07:10, CTD, 0, 0, Phytoplankton taxonomy]

["PolarNight2012","Rijpfjorden_m_2012-01-12",null,null,null]: Not sure what to do with this

["PolarNight2012","Sofiadypet_0m_2012-01-12",null,0,null] ["PolarNight2012","Sofiadypet_150m_2012-01-14",null,150,150] ["PolarNight2012","Sofiadypet_15m_2012-01-14",null,15,15] ["PolarNight2012","Sofiadypet_5m_2012-01-14",null,5,5] ["PolarNight2012","Sofiadypet_75m_2012-01-14",null,75,75] Four CTD taken from Sofiadypet / Ice station. If we choose the first CTD all samples could have the following info with different depths [PolarNight2012, Ice station, 81.6799, 14.3100, 2395, 14.01.2012, 12:23, 0, 0, Phytoplankton taxonomy]

I will also send the updated samplelog where the added records are in row 93-102

AnetteWold commented 2 years ago

Info missing samples N-ICE2015: ["N-ICE2015","PHT-144","inlogdoesnotexist",null,null]: Should be related to On-ice CTD-056 with depth 50m: [N-ICE2015, PHT-144, On-ice CTD-056, 56, 81.3710, 9.6187, 1412, 12.05.2015 07:33, On-ice CTD, 50]

["N-ICE2015","PHT-202","Floe3","in log does not exist",null]: Sampleset before (PHT196-201) and after (PHT203-208) looks complete. Does it exist data for all these IDs?

N-ICE2015,IAT-018 to IAT-030: We found some written notes from leg 2. Missing records sent by email (excell)

["N-ICE2015","IAT-041",null,null,null] Samplesets before (IAT33-40) & after (IAT42-49) looks complete. Does it exist data for all these IDs?

["N-ICE2015","IAT-072",null,null,null] Sampleset before (IAT66-71) & after (IAT74-76) looks complete. Does it exist data for all these IDs? ["N-ICE2015","IAT-073",null,null,null] same as above

["N-ICE2015","IAT-099",null,null,null]. Refrozen lead transect, could be one more point in the transect, but we do not have position, ice conditions etc for this point. ["N-ICE2015","IAT-100",null,null,null] same as above

["N-ICE2015","IAT-280",null,null,null] Could be part of Ridge thin ice transect, but we don't have any records of

cnrdh commented 2 years ago

Anette, I think we should move your comments into the relevant issue (for each expedition). OK?

AnetteWold commented 2 years ago

Ok

AnetteWold commented 2 years ago

MOSJ 2016: ["MOSJ2016","MIT-015","KB1_KB2",25,25]: There are two samples MIT-015a (Kb2) & MIT-015b (Kb1)

["MOSJ2016","HAN-018","R6",20,20]: According to the log the Handnet sample from R6 20-0m = HAN-17. Does it exist data for HAN-17? If it exist data for HAN-17 then add a record for HAN-18 for R7 [MOSJ2016, HAN-018, R7, 81.5115, 21.8475, 1046, 30.07.2016, 22:49, Handnet 20 μm, 20, 0, Handnet 20 μm, Philipp Assmy]

AnetteWold commented 2 years ago

GlacierFront2017 ["GlacierFront2017","PHT-031","CpS1",1,1]: Should be PHT-033 ["GlacierFront2017","HAN-KC4_Kb5","KC4_Kb5",20,20] Antar dette er Hand net samples som ikke er logget. KC4 ??? Kan legge til følgende [GlacierFront2017, HAN-KC4, Kb5, 78.8961, 12.4424, 80, 27.07.2017, 22:15, Niskin 20, 0, Phytoplankton taxonomy] Prøve fra Kb4 tilhører MOSJ og er registrert = MOSJ2017 HAN-001