glamod / glamod-ingest

Database preparation and ingestion for GLAMOD
BSD 2-Clause "Simplified" License
2 stars 1 forks source link

Fix matching of source_id when observation_id includes station_id with hyphen in it #48

Closed agstephens closed 3 years ago

agstephens commented 3 years ago

My code splits on the hyphen, so you are right, I might need to rerun everything.

I currently do this:

primary_id, record_number = x['observation_id'].split('-')[:2]

So, I would convert this as follows:

id = " BR00B6-0360-1-1965-01-01-44-13"
primary_id, record_number = id.split('-')[:2]
primary_id == 'BR00B6'
record_number == '0360'

What is the foolproof method of breaking it up?

Will that do the job?

agstephens commented 3 years ago

A new method of working back on the split would work, the files will always have :

Daily:  <YYYY>-<MM>-<DD>-<observed_variable>-<value_significance>
Sub daily : <YYYY>-<MM>-<DD>-<HH:MM>-<observed_variable>-<value_significance>
MONTHLY : <YYYY>-<MM>-<observed_variable>-<value_significance>
agstephens commented 3 years ago

Implemented a fix in: b7c21d1dc5d6d8ece7ed0e6a68de0fcb0abb9df0