datacamp / testwhat

Write Submission Correctness Tests for R exercises
https://datacamp.github.io/testwhat
GNU Affero General Public License v3.0
33 stars 25 forks source link

nested check_function() %>% check_arg() can fail #206

Closed richierocks closed 6 years ago

richierocks commented 6 years ago

Checking the arguments of a function inside another function is really useful for ggplots. I've come across a case that I think should work, but throws an error.

To reproduce

library(ggplot2)
library(testwhat)
code <- "ggplot(ChickWeight, aes(x = Time, y = weight)) + geom_line(aes(group = Chick))"
setup_state(code, code) %>% check_function("geom_line") %>% {
  check_arg(., "mapping") %>% check_function("aes") %>% {
    check_arg(., "group") %>% check_equal(eval = FALSE)
  }
}
##  Error in check_arg(., "group") : 
##    Make sure that the arguments you specify are actually specified by the 
##    corresponding function call in the solution code.

Here, I'm trying to check that geom_line() is called with it mapping argument calling aes(), which in turn has the group argument set to Chick.

It isn't clear to me why this fails.


Nesting these function can work; here's the same plot, but checking the ggplot part, and it works as expected.

library(ggplot2)
library(testwhat)
code <- "ggplot(ChickWeight, aes(x = Time, y = weight)) + geom_line(aes(group = Chick))"
setup_state(code, code) %>% check_function("ggplot") %>% {
  check_arg(., "mapping") %>% check_function("aes") %>% {
    check_arg(., "x") %>% check_equal(eval = FALSE)
    check_arg(., "y") %>% check_equal(eval = FALSE)
  }
}

Is this a testwhat bug or a problem with my SCT?

filipsch commented 6 years ago

@richierocks This could be because of a bug related to blacklisting. I want to spare you the details. I'm currently working on a PR that should simplify a lot of the logic, and maybe this automagically fixes it. I'll keep you posted.

filipsch commented 6 years ago

@richierocks We found the issue and will fix this soon! This is a pretty big bad bug!