Error rendering plots and reports

[johnbradley]

After upgrading to the latest code plots and reports are failing:

Error: An error has occurred. Check your logs or contact the app author for clarification.

In the pod logs there is this error.

Warning: Error in : could not find function "where"
--
  | 136: <Anonymous>

The where is part of tidyverse/tidyselect: https://tidyselect.r-lib.org/reference/language.html

[johnbradley]

The latest R 4.0.0 rocker/shiny-verse image has an upgraded tidyselect, but that version has issues with openshift: https://github.com/matthewhirschey/ddh/pull/89 OpenShift has stringent security requirements that the 4.0.0 image is not compatible with. There is an issue for this: https://github.com/rocker-org/rocker/issues/295

I am going to try just upgrading tidyselect using shiny-verse 3.6.3 to see if that will fix this problem.

[johnbradley]

It looks like we changed !any_of(c("cell_line", "lineage")) to where(is.numeric) in code/fun_plots.R: https://github.com/matthewhirschey/ddh/pull/97/files#diff-0adc94d148254eb981abb5a14b3cfde8

The version of R/tidyverse we are deploying doesn't include the where function. @matthewhirschey Do you know of a backwards compatible version of where(is.numeric)?

Instead of relying upon rocker-org as a base image I can try building the image from scratch, but it may take a little while.

[matthewhirschey]

working on it...

[matthewhirschey]

Try taking out where(), and just leaving: pivot_longer(cols = is.numeric, names_to = "gene_symbol", values_to = "dep_score") %>%

I get all sorts of warnings that this is deprecated, and where() is the current way to call a function within the select-helpers. So let's try that.

I just pushed a changed fun_plot.R file to the enhance_pathways branch, that you can test.

[johnbradley]

I deployed enhance_pathways but it now fails with:

Warning: Predicate functions must be wrapped in `where()`.
--
  |  
  | # Bad
  | data %>% select(is.numeric)
  |  
  | # Good
  | data %>% select(where(is.numeric))
  |  
  | ℹ Please update your code.
  | This message is displayed once per session.
  | Warning: Error in : This tidyselect interface doesn't support predicates yet.
  | ℹ Contact the package author and suggest using `eval_select()`.
  | 133: <Anonymous>

[matthewhirschey]

I got that as a warning too, but not an error to cause failure...

[johnbradley]

Yep Warning: Error is kind of confusing. If found a tidyverse_update function that recommends packages to update so I'm trying that out now.

[matthewhirschey]

OK. I also just pushed another change to enhance_pathways that should work. The code is very explicit, and I didn't like it as much as calling the numeric variables, but I think it'll work as-is.

[johnbradley]

I think the problem with reports might be a different issue. I saw this in the logs:

Warning: Error in ctx$onInvalidate: Reactive context was created in one process and accessed from another
--
  | 39: domain$onError
  | 38: promiseDomain$onError
  | 37: <Anonymous>
  | From earlier call:
  | 99: base$wrapOnRejected
  | 98: base$wrapOnRejected
  | 97: base$wrapOnRejected
  | 96: base$wrapOnRejected
  | 95: base$wrapOnRejected
  | 94: base$wrapOnRejected
  | 93: base$wrapOnRejected
  | 92: domain$wrapOnRejected
  | 91: promiseDomain$onThen
  | 90: action
  | 83: promise
  | 82: self$then
  | 81: promise$finally
  | 80: finally
  | 79: download$func [/srv/code/app.R#649]

[johnbradley]

Perhaps we need a call to force() https://github.com/rstudio/shiny/issues/2163 ?

[johnbradley]

I think the plots are working now after deploying the latest changes.

[matthewhirschey]

So where() is still there, because you upgraded tidy select, but not tidy verse, because that would have broken Rocker? In this case, I'm lost on that error. Never seen it. I can dig.

[johnbradley]

So where() is still there, because you upgraded tidy select, but not tidy verse, because that would have broken Rocker?

I think the problem I encountered after upgrading tidyselect was it needed other parts of tidyverse upgraded along with it. I upgraded the extra parts of tidyverse and we are now able to use where() without error. Ideally I would just upgrade to the latest rocker image (R 4) and start with an image that contained modern tidyverse. Unfortunately the latest rocker images are incompatible with OpenStack.

I think the error with the reports is a separate issue. Within RStudio I wasn't able to reproduce this problem. Running from the command line I was using : R -e "shiny::runApp('code')".

[johnbradley]

I think the problem with the report is we are accessing getQueryString() inside of the future function block. https://github.com/matthewhirschey/ddh/blob/7342f1f6e90f35c938f0314f3789b13db8da97a1/code/app.R#L628-L635

This needs to be done outside this block just like gene_symbol <- data().

We could setup some automated testing to catch these types of issues earlier.

[johnbradley]

Looks like report generation is still having issues. Example: https://www.datadrivenhypothesis.org/?show=detail&content=gene&symbol=BRCA1

Error in logs:

Warning in file(con, "w") :
--
  | cannot open file 'report_app.knit.md': Permission denied
  | Warning: Error in file: cannot open the connection
  | 39: domain$onError
  | 38: promiseDomain$onError
  | 37: <Anonymous>
  | From earlier call:
  | 99: base$wrapOnRejected
  | 98: base$wrapOnRejected
  | 97: base$wrapOnRejected
  | 96: base$wrapOnRejected
  | 95: base$wrapOnRejected
  | 94: base$wrapOnRejected
  | 93: base$wrapOnRejected
  | 92: domain$wrapOnRejected
  | 91: promiseDomain$onThen
  | 90: action
  | 83: promise
  | 82: self$then
  | 81: promise$finally
  | 80: finally
  | 79: download$func [/srv/code/app.R#650]

[matthewhirschey]

report_app.knit.md is an intermediate document before a PDF is generated Is it a problem with where/how this document is stored, which is then rendered to a PDF?

[johnbradley]

Is it a problem with where/how this document is stored, which is then rendered to a PDF?

I think that is the problem. The pdf generation writes files into the current directory. We have code to change to a temp directory which should be writable. Not sure why this is a problem again.

[matthewhirschey]

Everything checks out on my end; thanks for your help 🙏

[johnbradley]

Welcome. Always glad to help.