datacamp / testwhat

Write Submission Correctness Tests for R exercises
https://datacamp.github.io/testwhat
GNU Affero General Public License v3.0
33 stars 26 forks source link

Unexpected error with test_function(), test_correct(), and test_data_frame() #71

Closed tommyjee closed 8 years ago

tommyjee commented 8 years ago

@machow (cc @ncarchedi) Please see the SCTs corresponding to this exercise in Chapter 4 titled "Using gather to tidy a dataset". The link to the course on the teach app is here.

unspecified-4

I'm unsure why that commented-out bit throws an error. Could you help me out on this one?

machow commented 8 years ago

can you link to the repository for the course or paste in the code pieces (pre-exercise, solution, and sct)? I can't access it in the teach editor, and if it's a repo in the datacamp I'm not sure what its name is.

edit: nevermind, it looks like datacamp/courses-eda-unvotes!

filipsch commented 8 years ago

@tommyjee what's easiest for us is that you put in the source code in the github issue, so we don't have to go look for it!

tommyjee commented 8 years ago

@filipsch Okay, I'll do that in the future! @machow Here's the code: *\ =pre_exercise_code

source("http://s3.amazonaws.com/assets.datacamp.com/production/course_1414/datasets/shared.R")
votes_joined <- read_dataset("votes_joined")

*\ =solution

# Load the tidyr package
library(tidyr)

# Gather the six mu/nu/di/hr/co/ec columns
votes_joined %>%
  gather(topic, has_topic, me:ec)

# Perform gather again, then filter
votes_gathered <- votes_joined %>%
  gather(topic, has_topic, me:ec) %>%
  filter(has_topic == 1)

*\ =sct

test_library_function("tidyr")
# TODO: why does this throw an error?
# test_function("gather", args = c("data", "key", "value"))

# test_correct({
#     test_data_frame("votes_gathered", incorrect_msg = "Did you gather the six columns (`mu`, `nu`, `di`, `hr`, `co`, and `ec`) and filter such that `has_topic == 1` to create the `votes_gathered` dataset?")
#     }, {
#     test_function("gather", args = c("data", "key", "value"))
#     test_function("filter")
# })

test_error()
success_msg("Awesome job!")
machow commented 8 years ago

Thanks @tommyjee.

test_function doesn't work because the SCT

  1. runs the student and solution code
  2. tries to run each of args in the student's workspace to compare with the solution.

However, it hits an error because has_topic doesn't exist in the workspace (e.g. if you try to run has_topic after running the solution code it will fail). You can tell test_function to compare the arguments without trying to run the with the eval options, so using

test_function(gather, args = c('data', 'key', 'value'), eval=c(T, F, F))

will only evaluate the first argument, and compare that the arguments entered for the others match. In this case, if you change has_topic to has then test_function will mark it as incorrect.

I think the test_correct fails because behind the scenes it runs all the tests inside of it, but only fails if the first test argument fails. In this case, test_function is causing an error when run.

Will update the wiki with an explanation of why this happens. It's pretty common for R to use functions like substitute(mtcars, cyl > 6), where test_function would have no idea that cyl is inside mtcars.

tommyjee commented 8 years ago

Ah, good to know! I'll close this issue once I test it on the editor, just to be sure. Thanks @machow

cc @ncarchedi