`evalcast::download_signals()` issue (on `evalcast-killcards` branch)

[bnaras]

@dajmcdon The assert_that() call here assumes that empty_signals is a logical vector. But it is not as the attached log shows where I have used an instrumented version of the evalcast package with print statements to see what is going on. It turns out that empty_signals is a list of two elements, one of which is NULL, because out itself is also list of tibbles and a trailing NULL and sapply is unable to simplify (the s in sapply) and silently returns back a list. Lines 2750+ of the attached print values for inspection. run_zyzzyva.txt

[dajmcdon]

@sarah-colq This has to do with the fix you made the other day. Can you try to debug?

[sgsmob]

I can't replicate this. Below is the output of print(sapply(out, head)) and print(empty_signals) right after they are initialized.

            [,1]        [,2]        [,3]        [,4]       
data_source Character,6 Character,6 Character,6 Character,6
signal      Character,6 Character,6 Character,6 Character,6
geo_value   Character,6 Character,6 Character,6 Character,6
time_value  Numeric,6   Numeric,6   Numeric,6   Numeric,6  
issue       Numeric,6   Numeric,6   Numeric,6   Numeric,6  
lag         Integer,6   Integer,6   Integer,6   Integer,6  
value       Numeric,6   Numeric,6   Numeric,6   Numeric,6  
stderr      Logical,6   Logical,6   Numeric,6   Numeric,6  
sample_size Logical,6   Logical,6   Numeric,6   Logical,6  
[1] FALSE FALSE FALSE FALSE

What date are you using for the forecast date?

[sarah-colq]

Yeah, that's the type of output that's expected when I wrote this @sgsmob . I couldn't replicate it either.

[dajmcdon]

Naras and I just chatted on Zoom. It seems to be intermittent and hard to reproduce, possibly in covidcast rather than evalcast. The result he was getting from download_signals() was something like

ttt <- list(a=tibble(z=1), b=tibble(z=2), c=NULL)
sapply(ttt, function(x) nrow(x)==0) # old check
purrr::map_lgl(ttt, ~nrow(.x)==0 || is.null(.x)) # new check

The new check should catch it, but it's not clear why it happened in the first place.

tes <- covidcast::covidcast_signals(c("jhu-csse","usa-facts"),"confirmed_incidence_num", c("2020-01-01","2021-01-01"),c("2020-01-21","2021-01-21"),geo_type = "state")

will have a list with 2 components, the first of which is an empty tibble (no data for last January), but either the old or new checks work ok on it.

@bnaras just reran new zyzzyva on the production server with no errors, so there is output with a forecast date of tomorrow.

[dajmcdon]

Just had this happen. See #447 for a reprex (seems to happen in covidcast::aggregate_signals()).