Python - attempting to set station metadata B02002

[kruserr]

Hello,

I am attempting to set station metadata B02002, but it does not appear in the resulting csv. Am i doing something wrong here?

python snippet:

with self.db.transaction() as tr:
    tr.insert_station_data({
        'report': 'synop',
        'lat': station.lat,
        'lon': station.lon,
        'B01001': station.block_number,
        'B01002': station.station_number,
        'B02001': 0,
        'B02002': 8,
        'B07030': station.height,
        'B07031': station.barometer_height,
        'B01019': station.name,
    }, can_add_stations=True)

Resulting csv snippet:

"longitude","latitude","report","date","level","l","level","l","time rang","p","p","varcod","value"
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B01001,$REDACTED_BLOCK_NUMBER
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B01002,$REDACTED_STATION_NUMBER
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B01019,"$REDACTED_NAME"
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B02001,0
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B04001,2022
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B04002,2
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B04003,14
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B04004,9
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B04005,20
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B05001,$REDACTED_LAT
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B06001,$REDACTED_LON
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B07030,120.0
$REDACTED_LON,$REDACTED_LAT,"synop",,,,,,,,,B07031,99.9

Versions

• CentOS 7
• copr enable simc/stable epel-7 && yum install dballe
• dballe 9.0
• Python 3.6.8

[edigiacomo]

Hi @kruserr, I have tried to reproduce the problem.

Versions:

$ rpm -qi dballe | grep -i 'version\|release'
Version     : 9.0
Release     : 3.el7

$ python3 --version
Python 3.6.8

$ cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)

Simple script to import some data:

$ python3 <<EOF
> from datetime import datetime
> import dballe
> 
> 
> db = dballe.DB.connect("sqlite:tmp.db")
> db.reset()
> 
> 
> with db.transaction() as tr:
>     tr.insert_station_data({
>         'report': 'synop',
>         'lat': 42.12345,
>         'lon': 12.12345,
>         'B01001': 10,
>         'B01002': 11,
>         'B02001': 0,
>         'B02002': 8,
>         'B07030': 10,
>         'B07031': 10,
>         'B01019': "test",
>     }, can_add_stations=True)
> 
>     tr.insert_data({
>         'report': 'synop',
>         'lat': 42.12345,
>         'lon': 12.12345,
>         'B12101': 0,
>         'level': (103, 2000),
>         'trange': (254, 0, 0),
>         'datetime': datetime(2022, 2, 14),
>     })
> EOF

Var B02002 is present in the stations table:

$ dbadb stations --dsn=sqlite:tmp.db
#0: -----------------------
Station: 1,42.12345,12.12345 - synop
Var: 001001 WMO BLOCK NUMBER(NUMERIC): 10
Var: 001002 WMO STATION NUMBER(NUMERIC): 11
Var: 001019 LONG STATION OR SITE NAME(CCITTIA5): test
Var: 002001 TYPE OF STATION(CODE TABLE): 0
Var: 002002 TYPE OF INSTRUMENTATION FOR WIND MEASUREMENT(FLAG TABLE): 8
Var: 007030 HEIGHT OF STATION GROUND ABOVE MEAN SEA LEVEL (SEE NOTE 3)(M): 1.0
Var: 007031 HEIGHT OF BAROMETER ABOVE MEAN SEA LEVEL (SEE NOTE 4)(M): 1.0

Var B02002 is present in the generic template but it's removed from the synop:

$  dbadb export --dsn=sqlite:tmp.db | dbamsg dump --csv --interpreted
"longitude","latitude","report","date","level","l","level","l","time rang","p","p","varcod","value"
12.12345,42.12345,"synop",,,,,,,,,B01001,10
12.12345,42.12345,"synop",,,,,,,,,B01002,11
12.12345,42.12345,"synop",,,,,,,,,B01019,"test"
12.12345,42.12345,"synop",,,,,,,,,B02001,0
12.12345,42.12345,"synop",,,,,,,,,B04001,2022
12.12345,42.12345,"synop",,,,,,,,,B04002,2
12.12345,42.12345,"synop",,,,,,,,,B04003,14
12.12345,42.12345,"synop",,,,,,,,,B04004,0
12.12345,42.12345,"synop",,,,,,,,,B04005,0
12.12345,42.12345,"synop",,,,,,,,,B05001,42.12345
12.12345,42.12345,"synop",,,,,,,,,B06001,12.12345
12.12345,42.12345,"synop",,,,,,,,,B07030,1.0
12.12345,42.12345,"synop",,,,,,,,,B07031,1.0
12.12345,42.12345,"synop","2022-02-14 00:00:00",103,2000,,,254,0,0,B12101,0.00

$ dbadb export -t generic --dsn=sqlite:tmp.db | dbamsg dump --csv --interpreted
"longitude","latitude","report","date","level","l","level","l","time rang","p","p","varcod","value"
12.12345,42.12345,"synop",,,,,,,,,B01001,10
12.12345,42.12345,"synop",,,,,,,,,B01002,11
12.12345,42.12345,"synop",,,,,,,,,B01019,"test"
12.12345,42.12345,"synop",,,,,,,,,B01194,"synop"
12.12345,42.12345,"synop",,,,,,,,,B02001,0
12.12345,42.12345,"synop",,,,,,,,,B02002,8
12.12345,42.12345,"synop",,,,,,,,,B04001,2022
12.12345,42.12345,"synop",,,,,,,,,B04002,2
12.12345,42.12345,"synop",,,,,,,,,B04003,14
12.12345,42.12345,"synop",,,,,,,,,B04004,0
12.12345,42.12345,"synop",,,,,,,,,B04005,0
12.12345,42.12345,"synop",,,,,,,,,B04006,0
12.12345,42.12345,"synop",,,,,,,,,B05001,42.12345
12.12345,42.12345,"synop",,,,,,,,,B06001,12.12345
12.12345,42.12345,"synop",,,,,,,,,B07030,1.0
12.12345,42.12345,"synop",,,,,,,,,B07031,1.0
12.12345,42.12345,"synop","2022-02-14 00:00:00",103,2000,,,254,0,0,B12101,0.00

@dcesari @pat1 it's possibile that B02002 is a var not allowed in the synop template and therefore is removed from the output?

[pat1]

setting 'report': 'synop', set the network to a well know network; network exported to his default bufr template:

dbadb export  --template=list
acars - ACARS (autodetect)
acars-ecmwf - ACARS ECMWF (4.145)
acars-wmo - ACARS WMO
airep - AIREP (autodetect)
airep-ecmwf - AIREP ECMWF (4.142)
amdar - AMDAR (autodetect)
amdar-ecmwf - AMDAR ECMWF (4.144)
amdar-wmo - AMDAR WMO
buoy - Buoy (1.21)
generic - Generic (255.0)
metar - Metar (0.140)
pilot - pilot (autodetect)
pilot-ecmwf - Pilot (2.91)
pilot-wmo - Pilot (2.1, 2.2, 2.3)
pollution - Pollution (8.171)
ship - Synop ship (autodetect)
ship-abbr - Synop ship (abbreviated) (1.9)
ship-auto - Synop ship (auto) (1.13)
ship-plain - Synop ship (normal) (1.11)
ship-reduced - Synop ship (reduced) (1.19)
ship-second - Synop ship (second record) (1.12)
ship-wmo - Ship WMO
synop - Synop (autodetect)
synop-ecmwf - Synop ECMWF (autodetect) (0.1)
synop-ecmwf-auto - Synop ECMWF land auto (0.3)
synop-ecmwf-land - Synop ECMWF land (0.1)
synop-ecmwf-land-high - Synop ECMWF land high level station (0.1)
synop-wmo - Synop WMO (0.1)
temp - Temp (autodetect)
temp-ecmwf - Temp ECMWF (autodetect)
temp-ecmwf-land - Temp ECMWF land (2.101)
temp-ecmwf-ship - Temp ECMWF ship (2.102)
temp-radar - Temp radar doppler wind profile (6.1)
temp-ship - Temp ship (autodetect)
temp-wmo - Temp WMO (2.101)
wmo - WMO style templates (autodetect)

If you want export a "synop" network to a different template you have to specify it when export. The only template can export everything is "generic" template.

synop template do not support B02002 https://wmoomm.sharepoint.com/:w:/s/wmocpdb/EYZMR9DHlUNKs7XPpPY0a7wB8hiY4xRZPPfJzbndH-GQIw?rtime=PBbdMb3v2Ug and so it is removed