datacamp / testwhat

Write Submission Correctness Tests for R exercises
https://datacamp.github.io/testwhat
GNU Affero General Public License v3.0
33 stars 24 forks source link

check_function() should restrict the code that check_code() can find, but doesn't #209

Open richierocks opened 5 years ago

richierocks commented 5 years ago

Here's an example demonstrating the problem

https://www.datacamp.com/teach/repositories/986/branches/check_code-inside-check_function

hermansje commented 5 years ago

I think this is a duplicate of #206. The fixed testwhat is now deployed (so the demo should work, but it fails because dplyr isn't present).

richierocks commented 5 years ago

I think the correct way to do this is to find the subset of the parse data that corresponds to the function call, then reconstruct the code from that.

library(magrittr)
code <- "x <- mean(1:10)\ny <- sin(pi / 2)"
pd <- getParseData(parse(text = code))
pd %>% 
  getParseText(id = c(4, 5, 7, 9, 10, 12)) %>% 
  paste(collapse = "")
## [1] "mean(1:10)"

The tricky part is figuring out which IDs you need.

richierocks commented 5 years ago

OK, I think I got it.

get_function_call_id <- function(function_name, pd, index = 1L) {
  ids <- pd %>% 
    filter(token == "SYMBOL_FUNCTION_CALL", text == function_name) %>% 
    select(id) %>% 
    pull()
  ids[index]
}

get_parent_id <- function(ids, pd) {
  pd %>% filter(id %in% !!ids) %>% select(parent) %>% pull()
}

get_child_ids <- function(ids, pd) {
  pd %>% filter(parent %in% !!ids) %>% select(id) %>% pull()
}

get_descendent_ids <- function(ids, pd) {
  descendents <- integer()
  repeat{
    children <- get_child_ids(ids, pd)
    if(all(children %in% descendents)) {
      break
    }
    descendents <- c(descendents, children)
    ids <- children
  }
  descendents
}

reconstruct_code <- function(function_name, pd, index = 1L) {
  id_of_mean_function <- get_function_call_id(function_name, pd, index = index)
  grandparent <- id_of_mean_function %>% 
    get_parent_id(pd) %>% 
    get_parent_id(pd)
  descendents <- grandparent %>% 
    get_descendent_ids(pd)
  pd %>% 
    filter(terminal, id %in% descendents) %>% 
    select(text) %>% 
    pull() %>% 
    paste(collapse = "")
}

library(dplyr)
code <- "x <- mean(1:10)\ny <- mean(sin(pi / 2))"
pd <- getParseData(parse(text = code))
reconstruct_code("mean", pd)
## [1] "mean(1:10)"
reconstruct_code("mean", pd, 2)
[1] "mean(sin(pi/2))"
reconstruct_code("sin", pd)
## [1] "sin(pi/2)"
richierocks commented 5 years ago

This doesn't change .2 to 0.2 but it does strip some whitespace. That's likely a good idea though: if student code creates the same parse data as the solution, we should treat it the same.