search

GEUS-Glaciology-and-Climate / pyNEAD

Python interface for NEAD file format
GNU Lesser General Public License v3.0
1 stars 0 forks source link

Make nead.read OK with empty header lines #6

Closed BaptisteVandecrux closed 3 years ago

BaptisteVandecrux commented 3 years ago 
# [METADATA]
# doi = https://doi.org/10.16904/envidat.1
# reference = "Steffen, K. (2020). Greenland Climate Network (GC-Net) Data. EnviDat. doi: doi:10.16904/envidat.1"
# station_id = 107282
# station_name = GC-NET station Swiss Camp AWS
# srid = EPSG:4326
# geometry = POINTZ (61.0400 77.1377 1869)
# nodata = 
# timestamp_meaning = end
# timezone = 0
# field_delimiter = ,
# 
# [FIELDS]
# fields = timestamp,ISWR,ISWR_max,ISWR_std,OSWR,NSWR,NSWR_std,TA1,TA1_max,TA1_min,TA2,TA2_max,TA2_min,TA3,TA4,RH1,RH2,VW1,VW1_max,VW1_stdev,VW2,VW2_max,VW2_stdev,DW1,DW2,P,HS1,HS2,V,TA5
# add_value = 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
# scale_factor = 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0.01,0.01,1,1,1,1,1,1,1,1,100,1,1,1,1
# units = time,V,V,V,V,V,V,Degrees C,Degrees C,Degrees C,Degrees C,Degrees C,Degrees C,Degrees C,Degrees C,%,%,m/s,m/s,m/s,m/s,m/s,m/s,Degrees,Degrees,mbar,m,m,V,Degrees C
# display_description = timestamp_iso,shortwave_incoming_radiation,shortwave_incoming_radiation_max,shortwave_outgoing_radiation,shortwave_outgoing_radiation_min,net_radiation,net_radiation_max,air_temperature_1,air_temperature_1_max,air_temperature_1_min,air_temperature_2,air_temperature_2_max,air_temperature_2_min,air_temperature_cs500_air1,air_temperature_cs500_air2,relative_humidity_1,relative_humidity_2,wind_speed_1,wind_speed_u1_max,wind_speed_u1_stdev,wind_speed_2,wind_speed_u2_max,wind_speed_u2_stdev,wind_from_direction_1,wind_from_direction_2,air_pressure,snow_depth_1,snow_depth_2,battery_voltage,ref_temperature
# database_fields = timestamp_iso,swin,swin_maximum,swout,swout_minimum,netrad,netrad_maximum,airtemp1,airtemp1_maximum,airtemp1_minimum,airtemp2,airtemp2_maximum,airtemp2_minimum,airtemp_cs500air1,airtemp_cs500air2,rh1,rh2,windspeed1,windspeed_u1_maximum,windspeed_u1_stdev,windspeed2,windspeed_u2_maximum,windspeed_u2_stdev,winddir1,winddir2,pressure,sh1,sh2,battvolt,reftemp
# database_fields_data_types = timestamp,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real,real
# ​
# 
# [DATA]
# 
# 
2009-05-07 19:00:00,2.421,2.5069999999999997,0.048,5.5520000000000005,-0.866,0.11900000000000001,-13.19,-12.89,-13.54,-13.2,-12.98,-13.5,-11.93,-11.77,75.99,79.56,4.102,4.959,0.526,4.335,5.2589999999999995,0.5379999999999999,0.0,0.0,575.0,,,13.49,-7.882000000000001
2009-05-07 21:00:00,1.665,1.885,0.128,3.593,-0.616,0.064,-13.78,-13.29,-14.38,-13.69,-13.29,-14.22,-13.19,-12.95,79.9,83.4,4.45,5.416,0.45899999999999996,4.795,5.789,0.461,131.8,134.4,592.3,1.94,2.1390000000000002,13.23,-7.906000000000001
mankoff commented 3 years ago

Duplicate of #5?

mankoff commented 3 years ago

OK, not a duplicate, but I'm tempted to close this as wontfix. It sounds like you have an invalid input file. The NEAD specification defines the first line. Without it, it isn't a valid NEAD file.

BaptisteVandecrux commented 3 years ago

I now see that empty lines (with just '# ') are handled properly by the nead.read.

The problem was from ANSI character that was incidentally printed in the NEAD file I was trying to read and that were not appearing in the UTF-8 textfile.

What I see in UTF-8

# ​
# 
# [DATA]
# 
#

what it actually contains in ANSI:

# ​
# 
# [DATA]
# 
#

Just corrupted input file