Closed rjhd2 closed 2 years ago
Ah good catch ,I just set the minutes to 0 didn’t realise we had off hour obs. Will need to write and if else line of code for minutes .
Get Outlook for iOShttps://aka.ms/o0ukef
From: Robert Dunn @.> Sent: Monday, January 31, 2022 3:38:18 PM To: glamod/glamod-nuim @.> Cc: Simon Noone @.>; Assign @.> Subject: [EXTERNAL] [glamod/glamod-nuim] Fix minutes in Observation ID [Subdaily CDM Lite] (Issue #8)
Warning
This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
Input files have off-hour ovservations in the QFF:
FRI0000LFRI|LES AJONCS|2021|04|01|21|30|46.7019|-1.3786|91.1|15.0|223||ICAOLFRI-8|9.0|223||ICAOLFRI-8|||||||||50.0|223||ICAOLFRI-8|3.6|223||ICAOLFRI-8
But output CDM lite files only have on-hour values (no later values than 21:00 for this station on 2021-04-01 in the psv file):
FRI0000LFRI-2-2021-04-01-21:00-106-12|0|2021-04-01 21:00:00+00|1|46.702|-1.379|10|106|320|50|12|0||1|FRI0000LFRI|LES AJONCS|0|1|223
Uncomment in setting of master df
— Reply to this email directly, view it on GitHubhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fglamod%2Fglamod-nuim%2Fissues%2F8&data=04%7C01%7CSimon.Noone%40mu.ie%7C2a97b8d4476b45444df608d9e4cfb52b%7C1454f5ccbb354685bbd98621fd8055c9%7C0%7C0%7C637792403017072157%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2BZXV6ZLblAiMzSxeYgksvX%2BK3Firk6og9voD9%2BVVpk4%3D&reserved=0, or unsubscribehttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAG3VGPPMOGBWLDJZ7DDDVUDUY2UGVANCNFSM5NGS4H2Q&data=04%7C01%7CSimon.Noone%40mu.ie%7C2a97b8d4476b45444df608d9e4cfb52b%7C1454f5ccbb354685bbd98621fd8055c9%7C0%7C0%7C637792403017072157%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=og%2BieMZeKgPqvdX4eIrSFOXQ9zehaq2R1hi63fmUOtY%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7CSimon.Noone%40mu.ie%7C2a97b8d4476b45444df608d9e4cfb52b%7C1454f5ccbb354685bbd98621fd8055c9%7C0%7C0%7C637792403017072157%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=MV3qWwd5LiAirWteYJHti1ZKNhNeyIsbfU%2BmtGxGg5U%3D&reserved=0 or Androidhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7CSimon.Noone%40mu.ie%7C2a97b8d4476b45444df608d9e4cfb52b%7C1454f5ccbb354685bbd98621fd8055c9%7C0%7C0%7C637792403017072157%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=cyqEAF%2BgjZgsr8EIU07UESsYrnNmiAxvT1eqTAVKJm0%3D&reserved=0. You are receiving this because you were assigned.Message ID: @.***>
I think this edit will fix this bug in the cdm_lite and observations code existing df["Minute"]="00" edit df["Minute"]=df["Minute"]
Could we just remove the line df["Minute"]="00"
? I'm not sure what your edit achieves?
Your right actually Minutes is already in the DF from teh QFf so we can just remove it
Is the report_type
also needing to be carried through (the GWS files have it but this code currently overwrites with 0
)? If so, I'll amend the issue title and fix both in one go.
There is a rule in the recipe tables for the report type, I forgot about coding in for ICAO stations. [column 6]
Int report_type
Enter: 0=sub-daily/hourly data except for ICAO platforms 3=Daily data 2=monthly data 4= ICAO sub-daily data
Dr. Simon Noone
Senior Postdoctoral Researcher
https://orcid.org/0000-0003-1661-1423
https://www.maynoothuniversity.ie/icarus/our-people/simon-noone
The Copernicus Climate Change Service C3S2_311 - Collection and Processing of In Situ Observations. Lot 1 WP2-Lead Access to a comprehensive archive of historical surface observations, with support for data rescue.
@.**@.
Associated expert for the WMO Commission for Climatology Working Group on Climate Data Exchange (WG-CDE)
ICARUS - Irish Climate Research Centre Room 1.8,Laraghbryan House,
North campus, Maynooth University,
Maynooth, Kildare
Ireland
01-7086392 @. @.
From: Robert Dunn @.> Sent: Thursday, February 3, 2022 4:40:09 PM To: glamod/glamod-nuim @.> Cc: Simon Noone @.>; Assign @.> Subject: [EXTERNAL] Re: [glamod/glamod-nuim] Fix minutes in Observation ID [Subdaily CDM Lite] (Issue #8)
Warning
This email originated from outside of Maynooth University's Mail System. Do not reply, click links or open attachments unless you recognise the sender and know the content is safe.
Is the report_type also needing to be carried through (the GWS files have it but this code currently overwrites with 0)? If so, I'll amend the issue title and fix both in one go.
— Reply to this email directly, view it on GitHubhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fglamod%2Fglamod-nuim%2Fissues%2F8%23issuecomment-1029178887&data=04%7C01%7CSimon.Noone%40mu.ie%7Cbd724db0b7ae423436ce08d9e733d84c%7C1454f5ccbb354685bbd98621fd8055c9%7C0%7C0%7C637795032124942124%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ADkvxtQHMygsy5WOAIbI3qA2FX%2BgHEufz4GHSYgovWI%3D&reserved=0, or unsubscribehttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAG3VGPKZK24P6VUW7RHGBI3UZKVWTANCNFSM5NGS4H2Q&data=04%7C01%7CSimon.Noone%40mu.ie%7Cbd724db0b7ae423436ce08d9e733d84c%7C1454f5ccbb354685bbd98621fd8055c9%7C0%7C0%7C637795032124942124%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=o6kk%2BMIwm1EE2MRAtP%2BKEN9qes7QAiPMll4SMP0oaeo%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7CSimon.Noone%40mu.ie%7Cbd724db0b7ae423436ce08d9e733d84c%7C1454f5ccbb354685bbd98621fd8055c9%7C0%7C0%7C637795032124942124%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ZTTHSwVEkPKQFJ54UqEasb5WVbH1BVE5etirzbjWmoQ%3D&reserved=0 or Androidhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7CSimon.Noone%40mu.ie%7Cbd724db0b7ae423436ce08d9e733d84c%7C1454f5ccbb354685bbd98621fd8055c9%7C0%7C0%7C637795032124942124%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=brkWG1y7ViTTeuQvNW%2Ftb9jxAYo2Q7HQAE97lYHoIjs%3D&reserved=0. You are receiving this because you were assigned.Message ID: @.***>
Input files have off-hour ovservations in the QFF:
FRI0000LFRI|LES AJONCS|2021|04|01|21|30|46.7019|-1.3786|91.1|15.0|223||ICAOLFRI-8|9.0|223||ICAOLFRI-8|||||||||50.0|223||ICAOLFRI-8|3.6|223||ICAOLFRI-8
But output CDM lite files only have on-hour values (no later values than 21:00 for this station on 2021-04-01 in the psv file):
FRI0000LFRI-2-2021-04-01-21:00-106-12|0|2021-04-01 21:00:00+00|1|46.702|-1.379|10|106|320|50|12|0||1|FRI0000LFRI|LES AJONCS|0|1|223
Uncomment in setting of master
df