search

npolar / marine-db

https://doi.org/10.21334/marine-db
0 stars 0 forks source link

Samples log with multiple ids #4

Closed cnrdh closed 6 years ago

cnrdh commented 6 years ago 

{
    "sample": "AMM-016h; AMM-017h; AMM-018h",
    "expedition": "GlacierFront2017",
    "station": "KpM6",
    "sampled_from": "Helicopter",
    "cast": 5,
    "transect": "KpM",
    "gps": 11,
    "event": "340e8058-490c-5c11-8ef1-f05e847ea4b1"
}
cnrdh commented 6 years ago

Also note: extensive amount of extra metadata fields (not all shown here)

cnrdh commented 6 years ago

Quite a problem


$ cat data/master/sample/* | grep ";" | ndjson-filter 'd.sample.match(/;/)' | ndjson-map '[d.expedition,d.sample,d.time,d.gear]'
["GlacierFront2017","KpS5_R1; KpS5_R2","2017-07-25T08:53:00.000Z","Niskin"]
["GlacierFront2017","KpM5_R1_1M; KpM5_R2_1M","2017-07-25T13:25:00.000Z","Niskin"]
["GlacierFront2017","KpM5_R1_10M; KpM5_R2_10M","2017-07-25T13:25:00.000Z","Niskin"]
["GlacierFront2017","KpM5_R1_50M; KpM5_R2_50M","2017-07-25T13:25:00.000Z","Niskin"]
["GlacierFront2017","KpN5_R1_10M; KpN5_R2_10M","2017-07-25T20:40:00.000Z","Niskin"]
["GlacierFront2017","KpN4_R1_10M; KpN4_R2_10M","2017-07-26T07:45:00.000Z","Niskin"]
["GlacierFront2017","KpM4_R1_10M; KpM4_R1_10M","2017-07-26T10:40:00.000Z","Niskin"]
["GlacierFront2017","KpS4_R1_10M; KpS4_R2_10M","2017-07-26T12:00:00.000Z","Niskin"]
["GlacierFront2017","KpN2_R1_10M; KpN2_R2_10M","2017-07-27T08:00:00.000Z","Niskin"]
["GlacierFront2017","KpM2_R1_10M; KpM2_R2_10M","2017-07-27T10:10:00.000Z","Niskin"]
["GlacierFront2017","KpS2_R1_10M; KpS2_R2_10M","2017-07-27T11:30:00.000Z","Niskin"]
["GlacierFront2017","KpNS2_R1_10M; KpS2_R2_10M","2017-07-27T11:30:00.000Z","Niskin"]
["GlacierFront2017","Kc7_R1_10M; Kc7_R2_10M","2017-07-27T18:10:00.000Z","Niskin"]
["GlacierFront2017","Kc6_R1_10M; Kc6_R2_10M","2017-07-27T19:15:00.000Z","Niskin"]
["GlacierFront2017","Cc5_R1_10M; Cc5_R2_10M","2017-07-28T04:55:00.000Z","Niskin"]
["GlacierFront2017","Cc4_R1_10M; Cc4_R2_10M","2017-07-28T06:10:00.000Z","Niskin"]
["GlacierFront2017","Cc2_R1_10M; Cc2_R2_10M","2017-07-28T09:25:00.000Z","Niskin"]
["GlacierFront2017","CpN2_R1_10M; CpN2_R2_10M","2017-07-28T12:00:00.000Z","Niskin"]
["GlacierFront2017","CpN4_R1_10M;","2017-07-28T15:00:00.000Z","Niskin"]
["GlacierFront2017","CpS2_R1_10M; CpS2_R2_10M","2017-07-28T18:34:00.000Z","Niskin"]
["GlacierFront2017","CpS4_R1_10M;","2017-07-28T21:00:00.000Z","Niskin"]
["GlacierFront2017","AMM-007h; AMM-008h; AMM-009h","2017-07-25T11:25:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-004h; AMM-005h; AMM-006h","2017-07-25T11:25:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","URE-007; URE-008","2017-07-25T11:25:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","URE-005; URE-006","2017-07-25T11:25:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-016h; AMM-017h; AMM-018h","2017-07-25T13:06:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-013h; AMM-014h; AMM-015h","2017-07-25T13:06:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-010h; AMM-011h; AMM-012h","2017-07-25T13:06:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","URE-083;URE-084","2017-07-25T13:06:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","URE-081; URE-082","2017-07-25T13:06:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","URE-079; URE-088","2017-07-25T13:06:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-025h; AMM-026h; AMM-027h","2017-07-25T14:12:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-022h; AMM-023h; AMM-024h","2017-07-25T14:12:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","URE-89; URE-090","2017-07-25T14:12:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","URE-087; URE-088","2017-07-25T14:12:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-034h; AMM-035h; AMM-036h","2017-07-25T15:36:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-031h; AMM-032h; AMM-033h","2017-07-25T15:36:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-028h; AMM-029h; AMM-030h","2017-07-25T15:36:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","URE-095; URE-096","2017-07-25T15:36:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","URE-093; URE-094","2017-07-25T15:36:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","URE-091; URE-092","2017-07-25T15:36:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-043h; AMM-044h; AMM-045h","2017-07-26T11:38:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-040h; AMM-041h; AMM-042h","2017-07-26T11:38:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","AMM-001h; AMM-002h; AMM-003h","2017-07-26T11:38:00.000Z","Hydrobios mini CTD"]
["GlacierFront2017","URE-101; URE-102","2017-07-26T11:38:00.000Z","Hydrobios mini CTD"]
["MOSJ2017","KB3_R1_B;","2017-07-31T06:37:00.000Z","Niskin bottle"]
["MOSJ2017","KB3_R1_M; KB3_R2_M","2017-07-31T06:37:00.000Z","Niskin bottle"]
["MOSJ2017","KB3_R1_S; KB3_R2_S","2017-07-31T06:37:00.000Z","Niskin bottle"]
["MOSJ2017","KB0_R1_B;","2017-07-31T15:00:00.000Z","Niskin bottle"]
["MOSJ2017","KB0_R1_M; KB0_R2_M","2017-07-31T15:00:00.000Z","Niskin bottle"]
["MOSJ2017","KB0_R1_S; KB0_R2_S","2017-07-31T15:00:00.000Z","Niskin bottle"]
["MOSJ2017","V12_R1_B; V12_R1_B","2017-08-01T00:05:00.000Z","Niskin bottle"]
["MOSJ2017","V12_R1_M; V12_R1_M","2017-08-01T00:05:00.000Z","Niskin bottle"]
["MOSJ2017","V12_R1_25m; V12_R1_25m","2017-08-01T00:05:00.000Z","Niskin bottle"]
["MOSJ2017","V12_R1_S; V12_R1_S","2017-08-01T00:05:00.000Z","Niskin bottle"]
["MOSJ2017","V3_R1_1000M; V6_R2_1000M","2917-08-01T09:33:00.000Z","Niskin bottle"]
["MOSJ2017","V3_R1_B; V6_R2_B","2917-08-01T09:33:00.000Z","Niskin bottle"]
["MOSJ2017","V6_R1_M; V6_R2_M","2917-08-01T09:33:00.000Z","Niskin bottle"]
["MOSJ2017","V6_R1_S; V6_R2_S","2917-08-01T09:33:00.000Z","Niskin bottle"]
["MOSJ2017","HGIV_R1_50m; HGIV_R2_50m","2017-08-02T08:39:00.000Z","Niskin bottle"]
["MOSJ2017","HGIV_R1_25m; HGIV_R2_25m","2017-08-02T08:39:00.000Z","Niskin bottle"]
["MOSJ2017","HGIV_R1_10m; HGIV_R2_10m","2017-08-02T08:39:00.000Z","Niskin bottle"]
["MOSJ2017","R1_R1_B; R1_R2_B","2017-08-03T13:52:00.000Z","Niskin bottle"]
["MOSJ2017","R1_R1_M; R1_R2_M","2017-08-03T13:52:00.000Z","Niskin bottle"]
["MOSJ2017","R1_R1_S; R1_R2_S","2017-08-03T13:52:00.000Z","Niskin bottle"]
["MOSJ2017","R3_R1_B; R3_R2_B","2017-08-03T19:54:00.000Z","Niskin bottle"]
["MOSJ2017","R3_R1_M; R3_R2_M","2017-08-03T19:54:00.000Z","Niskin bottle"]
["MOSJ2017","R3_R1_S; R3_R2_S","2017-08-03T19:54:00.000Z","Niskin bottle"]
["MOSJ2017","R6_R1_B; R6_R2_B","2017-08-04T10:00:00.000Z","Niskin bottle"]
["MOSJ2017","R6_R1_M; R6_R2_M","2017-08-04T10:00:00.000Z","Niskin bottle"]
["MOSJ2017","R6_R1_S; R6_R2_S","2017-08-04T10:00:00.000Z","Niskin bottle"]
["MOSJ2017","R6_R1_B; R6_R2_B","2017-08-04T17:00:00.000Z","Niskin bottle"]
["MOSJ2017","R6_R1_M; R6_R2_M","2017-08-04T17:00:00.000Z","Niskin bottle"]
["MOSJ2017","R6_R1_S; R6_R2_S","2017-08-04T17:00:00.000Z","Niskin bottle"]
["MOSJ2017","R6b_R1_B; R6b_R2_B","2017-08-04T21:30:00.000Z","Niskin bottle"]
["MOSJ2017","R6b_R1_M; R6b_R2_M","2017-08-04T21:30:00.000Z","Niskin bottle"]
["MOSJ2017","R6b_R1_S; R6b_R2_S","2017-08-04T21:30:00.000Z","Niskin bottle"]
["MOSJ2017","R8_R1_890m; R8_R2_890m","2017-08-05T15:30:00.000Z","Niskin bottle"]
["MOSJ2017","R8_R1_100m; R8_R2_100m","2017-08-05T15:30:00.000Z","Niskin bottle"]
["MOSJ2017","R8_R1_50m; R8_R2_50m","2017-08-05T15:30:00.000Z","Niskin bottle"]
["MOSJ2017","R8_R1_15m; R8_R2_15m","2017-08-05T15:30:00.000Z","Niskin bottle"]
["MOSJ2017","R8_R1_5m; R8_R2_5m","2017-08-05T15:30:00.000Z","Niskin bottle"]
cnrdh commented 6 years ago

And also: ";" is not the only in-cell separator; "," is also used...

cat data/master/sample/* | ndjson-map 'd.sample' | grep ","
"OTI03_EE4,2_WP3"
"OTI03_EE4,2_MPS"

2003-08-20T14:30:00Z            EE4,2   Lance   OTI2003 "WP3 1000 µm"   "OTI03 EE4,2 WP3"   340-0   taxonomy    mesozooplankton 350 
2003-08-20T15:05:00Z            EE4,2   Lance   OTI2003 "Multinet 200 µm"   "OTI03 EE4,2 MPS"   330-0   taxonomy    mesozooplankton 352

Fixed by using _ for comma, cannot find any references to these samples...