search

timriffe / covid_age

COVerAGE-DB: COVID-19 cases, deaths, and tests by age and sex
Other
56 stars 30 forks source link

Possibly wrong new year assigned in input - Wisconsin case #68

Closed mpascariu closed 3 years ago

mpascariu commented 3 years ago

Hi @timriffe

I can see this:

coverage <- read_csv(
  file = "data/Output_10_20210107.zip",
  skip = 3) %>% 
  filter(Region == "Wisconsin",
         Sex == "b") %>% 
  mutate(Date = as.Date(Date, format = "%d.%m.%Y")) %>% 
  arrange(Date) %>% 
  print(n = 70)

# A tibble: 3,124 x 10
   Country Region    Code             Date         Age AgeInt Sex     Cases Deaths Tests
   <chr>   <chr>     <chr>            <date>     <dbl>  <dbl> <chr>   <dbl>  <dbl> <lgl>
 1 USA     Wisconsin US_WI_01.01.2020 2020-01-01     0     10 b     18816      0   NA   
 2 USA     Wisconsin US_WI_01.01.2020 2020-01-01    10     10 b     54477      2   NA   
 3 USA     Wisconsin US_WI_01.01.2020 2020-01-01    20     10 b     92503     16   NA   
 4 USA     Wisconsin US_WI_01.01.2020 2020-01-01    30     10 b     75498     37   NA   
 5 USA     Wisconsin US_WI_01.01.2020 2020-01-01    40     10 b     68817     82   NA   
 6 USA     Wisconsin US_WI_01.01.2020 2020-01-01    50     10 b     73912    268   NA   
 7 USA     Wisconsin US_WI_01.01.2020 2020-01-01    60     10 b     52605    641   NA   
 8 USA     Wisconsin US_WI_01.01.2020 2020-01-01    70     10 b     27593   1217   NA   
 9 USA     Wisconsin US_WI_01.01.2020 2020-01-01    80     10 b     13546   1539   NA   
10 USA     Wisconsin US_WI_01.01.2020 2020-01-01    90     10 b      5216.  1060.  NA   
11 USA     Wisconsin US_WI_01.01.2020 2020-01-01   100      5 b        24.2    7.5 NA   
12 USA     Wisconsin US_WI_02.01.2020 2020-01-02     0     10 b     18864      0   NA   
13 USA     Wisconsin US_WI_02.01.2020 2020-01-02    10     10 b     54588      2   NA   
14 USA     Wisconsin US_WI_02.01.2020 2020-01-02    20     10 b     92705     16   NA   
15 USA     Wisconsin US_WI_02.01.2020 2020-01-02    30     10 b     75672     37   NA   
16 USA     Wisconsin US_WI_02.01.2020 2020-01-02    40     10 b     68979     82   NA   
17 USA     Wisconsin US_WI_02.01.2020 2020-01-02    50     10 b     74056    268   NA   
18 USA     Wisconsin US_WI_02.01.2020 2020-01-02    60     10 b     52729    641   NA   
19 USA     Wisconsin US_WI_02.01.2020 2020-01-02    70     10 b     27662   1217   NA   
20 USA     Wisconsin US_WI_02.01.2020 2020-01-02    80     10 b     13581   1540   NA   
21 USA     Wisconsin US_WI_02.01.2020 2020-01-02    90     10 b      5225.  1060.  NA   
22 USA     Wisconsin US_WI_02.01.2020 2020-01-02   100      5 b        24.2    7.5 NA   
23 USA     Wisconsin US_WI_03.01.2020 2020-01-03     0     10 b     19036      0   NA   
24 USA     Wisconsin US_WI_03.01.2020 2020-01-03    10     10 b     54876      2   NA   
25 USA     Wisconsin US_WI_03.01.2020 2020-01-03    20     10 b     93085     16   NA   
26 USA     Wisconsin US_WI_03.01.2020 2020-01-03    30     10 b     76059     37   NA   
27 USA     Wisconsin US_WI_03.01.2020 2020-01-03    40     10 b     69299     82   NA   
28 USA     Wisconsin US_WI_03.01.2020 2020-01-03    50     10 b     74394    269   NA   
29 USA     Wisconsin US_WI_03.01.2020 2020-01-03    60     10 b     53018    642   NA   
30 USA     Wisconsin US_WI_03.01.2020 2020-01-03    70     10 b     27801   1217   NA   
31 USA     Wisconsin US_WI_03.01.2020 2020-01-03    80     10 b     13685   1542   NA   
32 USA     Wisconsin US_WI_03.01.2020 2020-01-03    90     10 b      5254.  1060.  NA   
33 USA     Wisconsin US_WI_03.01.2020 2020-01-03   100      5 b        24.2    7.5 NA   
34 USA     Wisconsin US_WI_04.01.2020 2020-01-04     0     10 b     19139      0   NA   
35 USA     Wisconsin US_WI_04.01.2020 2020-01-04    10     10 b     55072      2   NA   
36 USA     Wisconsin US_WI_04.01.2020 2020-01-04    20     10 b     93318     16   NA   
37 USA     Wisconsin US_WI_04.01.2020 2020-01-04    30     10 b     76268     37   NA   
38 USA     Wisconsin US_WI_04.01.2020 2020-01-04    40     10 b     69467     82   NA   
39 USA     Wisconsin US_WI_04.01.2020 2020-01-04    50     10 b     74581    269   NA   
40 USA     Wisconsin US_WI_04.01.2020 2020-01-04    60     10 b     53153    642   NA   
41 USA     Wisconsin US_WI_04.01.2020 2020-01-04    70     10 b     27877   1219   NA   
42 USA     Wisconsin US_WI_04.01.2020 2020-01-04    80     10 b     13758   1545   NA   
43 USA     Wisconsin US_WI_04.01.2020 2020-01-04    90     10 b      5281.  1064.  NA   
44 USA     Wisconsin US_WI_04.01.2020 2020-01-04   100      5 b        24.3    7.5 NA   
45 USA     Wisconsin US_WI_05.01.2020 2020-01-05     0     10 b     19279      0   NA   
46 USA     Wisconsin US_WI_05.01.2020 2020-01-05    10     10 b     55447      2   NA   
47 USA     Wisconsin US_WI_05.01.2020 2020-01-05    20     10 b     93869     16   NA   
48 USA     Wisconsin US_WI_05.01.2020 2020-01-05    30     10 b     76818     37   NA   
49 USA     Wisconsin US_WI_05.01.2020 2020-01-05    40     10 b     69972     84   NA   
50 USA     Wisconsin US_WI_05.01.2020 2020-01-05    50     10 b     75131    278   NA   
51 USA     Wisconsin US_WI_05.01.2020 2020-01-05    60     10 b     53543    655   NA   
52 USA     Wisconsin US_WI_05.01.2020 2020-01-05    70     10 b     28091   1241   NA   
53 USA     Wisconsin US_WI_05.01.2020 2020-01-05    80     10 b     13850   1572   NA   
54 USA     Wisconsin US_WI_05.01.2020 2020-01-05    90     10 b      5316.  1086.  NA   
55 USA     Wisconsin US_WI_05.01.2020 2020-01-05   100      5 b        24.5    7.7 NA   
56 USA     Wisconsin US_WI_29.03.2020 2020-03-29     0     10 b         4      0   NA   
57 USA     Wisconsin US_WI_29.03.2020 2020-03-29    10     10 b        14      0   NA   
58 USA     Wisconsin US_WI_29.03.2020 2020-03-29    20     10 b       148      0   NA   
59 USA     Wisconsin US_WI_29.03.2020 2020-03-29    30     10 b       169      0   NA   
60 USA     Wisconsin US_WI_29.03.2020 2020-03-29    40     10 b       186      0   NA   
61 USA     Wisconsin US_WI_29.03.2020 2020-03-29    50     10 b       203      4   NA   
62 USA     Wisconsin US_WI_29.03.2020 2020-03-29    60     10 b       220      3   NA   
63 USA     Wisconsin US_WI_29.03.2020 2020-03-29    70     10 b       111      3   NA   
64 USA     Wisconsin US_WI_29.03.2020 2020-03-29    80     10 b        46      2   NA   
65 USA     Wisconsin US_WI_29.03.2020 2020-03-29    90     10 b        10.9    1   NA   
66 USA     Wisconsin US_WI_29.03.2020 2020-03-29   100      5 b         0.1    0   NA   
67 USA     Wisconsin US_WI_30.03.2020 2020-03-30     0     10 b         4      0   NA   
68 USA     Wisconsin US_WI_30.03.2020 2020-03-30    10     10 b        16      0   NA   
69 USA     Wisconsin US_WI_30.03.2020 2020-03-30    20     10 b       161      0   NA   
70 USA     Wisconsin US_WI_30.03.2020 2020-03-30    30     10 b       179      0   NA   
# ... with 3,054 more rows

Is it possible that the wrong year is assigned to first 5 recorded days? 2020 instead of 2021? Note the jump in at line 56, from January to March.

timriffe commented 3 years ago

Thanks @kikeacosta is adjusting the script just now. If you see more let us know. We realized yesterday some scripts (older ones) were hard coded to 2020, shouldn't take long to take care of this.

mpascariu commented 3 years ago

Hi @kikeacosta and @timriffe This issue can still be seen for the state of Oregon, however only for January 4.

library(lubridate)
library(tidyverse)

coverage <- read_csv(
  file = "data/Output_10_20210108.zip",
  skip = 3) %>% 
  mutate(Date = as.Date(Date, format = "%d.%m.%Y"),
         mth = month(Date)) %>% 
  filter(Region == "Oregon",
         Sex == "b",
         mth == 1) %>% 
  arrange(Date) %>% 
  print(n = Inf)

# A tibble: 33 x 11
   Country Region Code            Date         Age AgeInt Sex     Cases Deaths Tests   mth
   <chr>   <chr>  <chr>           <date>     <dbl>  <dbl> <chr>   <dbl>  <dbl> <lgl> <dbl>
 1 USA     Oregon US_OR04.01.2020 2020-01-04     0     10 b      5505.     0   NA        1
 2 USA     Oregon US_OR04.01.2020 2020-01-04    10     10 b     12562.     0   NA        1
 3 USA     Oregon US_OR04.01.2020 2020-01-04    20     10 b     24968.     2   NA        1
 4 USA     Oregon US_OR04.01.2020 2020-01-04    30     10 b     20933.    13   NA        1
 5 USA     Oregon US_OR04.01.2020 2020-01-04    40     10 b     18616.    27   NA        1
 6 USA     Oregon US_OR04.01.2020 2020-01-04    50     10 b     15220.    93   NA        1
 7 USA     Oregon US_OR04.01.2020 2020-01-04    60     10 b     10112.   214   NA        1
 8 USA     Oregon US_OR04.01.2020 2020-01-04    70     10 b      5970.   373   NA        1
 9 USA     Oregon US_OR04.01.2020 2020-01-04    80     10 b      2871.   387.  NA        1
10 USA     Oregon US_OR04.01.2020 2020-01-04    90     10 b      1679    390.  NA        1
11 USA     Oregon US_OR04.01.2020 2020-01-04   100      5 b        17.6    6.4 NA        1
12 USA     Oregon US_OR05.01.2021 2021-01-05     0     10 b      5545.     0   NA        1
13 USA     Oregon US_OR05.01.2021 2021-01-05    10     10 b     12662.     0   NA        1
14 USA     Oregon US_OR05.01.2021 2021-01-05    20     10 b     25199.     2   NA        1
15 USA     Oregon US_OR05.01.2021 2021-01-05    30     10 b     21101.    13   NA        1
16 USA     Oregon US_OR05.01.2021 2021-01-05    40     10 b     18772.    29   NA        1
17 USA     Oregon US_OR05.01.2021 2021-01-05    50     10 b     15359     98   NA        1
18 USA     Oregon US_OR05.01.2021 2021-01-05    60     10 b     10202.   221   NA        1
19 USA     Oregon US_OR05.01.2021 2021-01-05    70     10 b      6031.   381   NA        1
20 USA     Oregon US_OR05.01.2021 2021-01-05    80     10 b      2902.   397.  NA        1
21 USA     Oregon US_OR05.01.2021 2021-01-05    90     10 b      1696.   402.  NA        1
22 USA     Oregon US_OR05.01.2021 2021-01-05   100      5 b        17.8    6.7 NA        1
23 USA     Oregon US_OR06.01.2021 2021-01-06     0     10 b      5571.     0   NA        1
24 USA     Oregon US_OR06.01.2021 2021-01-06    10     10 b     12726.     0   NA        1
25 USA     Oregon US_OR06.01.2021 2021-01-06    20     10 b     25337.     2   NA        1
26 USA     Oregon US_OR06.01.2021 2021-01-06    30     10 b     21256.    13   NA        1
27 USA     Oregon US_OR06.01.2021 2021-01-06    40     10 b     18898.    29   NA        1
28 USA     Oregon US_OR06.01.2021 2021-01-06    50     10 b     15453.    99   NA        1
29 USA     Oregon US_OR06.01.2021 2021-01-06    60     10 b     10266    224   NA        1
30 USA     Oregon US_OR06.01.2021 2021-01-06    70     10 b      6071.   382   NA        1
31 USA     Oregon US_OR06.01.2021 2021-01-06    80     10 b      2921.   397.  NA        1
32 USA     Oregon US_OR06.01.2021 2021-01-06    90     10 b      1706.   405.  NA        1
33 USA     Oregon US_OR06.01.2021 2021-01-06   100      5 b        17.8    6.8 NA        1
timriffe commented 3 years ago

fixed!