bcgov / envair

BC ENV Air Quality Monitoring Data
Apache License 2.0
6 stars 4 forks source link

duplicate rows #7

Closed GailR closed 4 years ago

GailR commented 4 years ago

Hi @jeromerobles

importBC_data() generates 8 duplicate timestamps at the beginning of each year.

I believe this is an importBC_data() issue as the original data does not contain the duplicate rows. Data: ftp://ftp.env.gov.bc.ca//pub/outgoing/AIR/AnnualSummary/2018/STATION_DATA/E295892.csv

I wonder if this issue is related to #4? There there is an 8-hour difference between UTC and PST.

farm <- importBC_data('Farmington', 2018)
farm %>% 
  select(date, station_name, ems_id, contains("o3")) %>% 
  arrange(date)

                  date              station_name  ems_id    o3_raw   o3 o3_instrument
1  2018-01-01 00:00:00 Farmington Community Hall E295892        NA   NA          <NA>
2  2018-01-01 00:00:00 Farmington Community Hall E295892  6.848054  6.8    O3_APIT400
3  2018-01-01 01:00:00 Farmington Community Hall E295892        NA   NA          <NA>
4  2018-01-01 01:00:00 Farmington Community Hall E295892  6.640554  6.6    O3_APIT400
5  2018-01-01 02:00:00 Farmington Community Hall E295892        NA   NA          <NA>
6  2018-01-01 02:00:00 Farmington Community Hall E295892  5.903611  5.9    O3_APIT400
7  2018-01-01 03:00:00 Farmington Community Hall E295892        NA   NA          <NA>
8  2018-01-01 03:00:00 Farmington Community Hall E295892  4.670833  4.7    O3_APIT400
9  2018-01-01 04:00:00 Farmington Community Hall E295892        NA   NA          <NA>
10 2018-01-01 04:00:00 Farmington Community Hall E295892  5.527222  5.5    O3_APIT400
11 2018-01-01 05:00:00 Farmington Community Hall E295892        NA   NA          <NA>
12 2018-01-01 05:00:00 Farmington Community Hall E295892  9.448887  9.4    O3_APIT400
13 2018-01-01 06:00:00 Farmington Community Hall E295892        NA   NA          <NA>
14 2018-01-01 06:00:00 Farmington Community Hall E295892 33.229170 33.2    O3_APIT400
15 2018-01-01 07:00:00 Farmington Community Hall E295892        NA   NA          <NA>
16 2018-01-01 07:00:00 Farmington Community Hall E295892 36.935830 36.9    O3_APIT400
17 2018-01-01 08:00:00 Farmington Community Hall E295892 37.506110 37.5    O3_APIT400
18 2018-01-01 09:00:00 Farmington Community Hall E295892 39.153610 39.2    O3_APIT400
jeromerobles commented 4 years ago

Issue may have been resolved a few iterations ago. I am unable to replicate. Can you update to the latest version and check? remotes::install_github("bcgov/envair") library(envair) farm <- importBC_data('Farmington', 2018) farm %>% select(date, station_name, ems_id, contains("o3")) %>% arrange(date)

date station_name ems_id o3_raw o3 o3_instrument

1 2018-01-01 00:00:00 Farmington Community Hall E295892 6.85 6.8 O3_APIT400 2 2018-01-01 01:00:00 Farmington Community Hall E295892 6.64 6.6 O3_APIT400 3 2018-01-01 02:00:00 Farmington Community Hall E295892 5.90 5.9 O3_APIT400 4 2018-01-01 03:00:00 Farmington Community Hall E295892 4.67 4.7 O3_APIT400 5 2018-01-01 04:00:00 Farmington Community Hall E295892 5.53 5.5 O3_APIT400 6 2018-01-01 05:00:00 Farmington Community Hall E295892 9.45 9.4 O3_APIT400 7 2018-01-01 06:00:00 Farmington Community Hall E295892 33.2 33.2 O3_APIT400 8 2018-01-01 07:00:00 Farmington Community Hall E295892 36.9 36.9 O3_APIT400 9 2018-01-01 08:00:00 Farmington Community Hall E295892 37.5 37.5 O3_APIT400 10 2018-01-01 09:00:00 Farmington Community Hall E295892 39.2 39.2 O3_APIT400
GailR commented 4 years ago

Thanks @jeromerobles. This issue has been resolved. I have been able to replicate your result on both mac and pc. I thought I had the newest version installed when originally posting this issue but apparently not!