search

aodn / public-schema

Shared schema specifications for data exchange (WFS, CSV, etc...)
GNU General Public License v3.0
1 stars 0 forks source link

CSIRO Geoserver layers to be made consistent with schema #22

Closed mhidas closed 2 years ago

mhidas commented 2 years ago

The Geoserver layers for bgc_chemistry and bgc_pigments are currently incosistent with the schemas defined here (see test results - I think it's just the PROJECTNAME column that needs to be removed).

Please update these when possible.

mhidas commented 2 years ago

Also, the bgc_chemistry table seems to be corrupted when downloaded to a csv file.

The initial lines look fine:

PROJECTNAME,TRIP_CODE,SAMPLEDEPTH_M,SAMPLEDATELOCAL,SALINITY_PSU,SALINITY_FLAG,SALINITY_COMMENTS,SILICATE_UMOLL,SILICATE_FLAG,NITRATE_UMOLL,NITRATE_FLAG,PHOSPHATE_UMOLL,PHOSPHATE_FLAG,AMMONIUM_UMOLL,AMMONIUM_FLA
G,NITRITE_UMOLL,NITRITE_FLAG,NUTRIENT_COMMENTS,DIC_UMOLKG,CARBON_FLAG,CARBON_COMMENTS,TALKALINITY_UMOLKG,ALKALINITY_FLAG,ALKALINITY_COMMENTS,OXYGEN_UMOLL,OXYGEN_FLAG,OXYGEN_COMMENTS,MICROBIOMESAMPLE_ID
Coastal Stations,BAI20200618,2,2020-06-18 00:44:00,,9,,2.11666667,1,2.38333333,1,0.305,1,1.305,1,0.21833333,1,,,9,,,9,,,9,,
Coastal Stations,BAI20200716,2,2020-07-16 00:56:00,,9,,0.96666667,1,1.45,1,0.25333333,1,1.05166667,1,0.18333333,1,,,9,,,9,,,9,,
Coastal Stations,BAI20200827,2,2020-08-27 00:05:00,,9,,0.6,1,0.78333333,1,0.18,1,0.48833333,1,0.15166667,1,,,9,,,9,,,9,,
...

but there are a few rows that look like this:

NRS,YON20111027,10,2011-10-27 13:59:00,,9,"no salinity; bottle broken
0.1,1,0,1,0.06,1,0.07,1,,9,,1997.72,1,,2307.29,1,,,9,,

The problem is the SALINITY_COMMENTS column starts with a double-quote character (with no matching one at the end) and seems to end with a newline character. This tends to wreak having with software trying to parse the file.