search

dynverse / dyno

Inferring, interpreting and visualising trajectories using a streamlined set of packages 🦕
https://dynverse.github.io/dyno
Other
170 stars 32 forks source link

error : $ operator is invalid for atomic vectors #40

Open davisidarta opened 5 years ago

davisidarta commented 5 years ago

I've been trying to run Dyno so as to proceed to lineage inference from a public dataset.

I've done the preprocessing to reshape de data to the required input format (counts and expression matrices with cells as rows and genes as columns). I've assigned cells IDs and genes as rownames and colnames.

When trying to run PAGA, the following error pops out: $ operator is invalid for atomic vectors

Here's my piece of code so far. Any insights on why this is happening? Thanks in advance!

Create a dyno dataset with wrap_expression

library(tidyverse) library(dyno)

dataset <- wrap_expression(counts = counts, expression = exp)

Run dynguidelines

guidelines <- guidelines_shiny(dataset)

answers <- dynguidelines::answer_questions( multiple_disconnected = NULL, expect_topology = NULL, expected_topology = NULL, n_cells = 19743, n_features = 20921, memory = "8GB", prior_information = c("start_id", "end_id", "end_n", "start_n", "groups_n", "features_id"), docker = TRUE )

Adding prior information to the dataset (PAGA needs it to run)

dataset <- add_prior_information(dataset, start_id = "arc1_CAAATCACAAGC")

Reproduces the guidelines as created in the shiny app

guidelines <- dynguidelines::guidelines(answers = answers)

methods_selected <- guidelines$methods_selected

Run the selected method

model <- infer_trajectory(dataset, first(methods_selected))

As an output from the last line, I get:

model <- infer_trajectory(dataset, first(methods_selected)) Error traceback: 1: Killed

Error: $ operator is invalid for atomic vectors

haircell commented 5 years ago

Not sure if this is the same, but I also had trouble running this line with the tutorial/vignette: model <- infer_trajectory(dataset, first(methods_selected))

Got around it by first listing the "methods_selected" then just choosing which method to run:

methods_selected <- guidelines$methods_selected
methods_selected 
model <- infer_trajectory(dataset, "your chosen method")
zouter commented 5 years ago

Hi @davisidarta and @haircell Could you try to install the devel version of dynwrap? devtools::install_github("dynverse/dynwrap@devel")

This contains a hotfix that should fix this error.

dallen59 commented 5 years ago

Hi @zouter ,

I have also been having the same problem with PAGA. I installed the devel version but now I get this error when I try to run PAGA:

model_PAGA <- infer_trajectory(dataset = dataset, method = "paga", verbose = T) 

Error in infer_trajectory(dataset = dataset, method = "paga", verbose = T) : 
  lazy-load database '/home/dallen/R/x86_64-pc-linux-gnu-library/3.5/dynwrap/R/dynwrap.rdb' is corrupt
In addition: Warning message:
In infer_trajectory(dataset = dataset, method = "paga", verbose = T) :
  internal error -3 in R_decompress1
zouter commented 5 years ago

Hi everyone

This error message originated from an error within the wrappers of the TI methods. However, we captured these errors incorrectly, and because of that you got this strange error Error: $ operator is invalid for atomic vectors.

I pushed a fix to dynverse/dynwrap to resolve this. It will now print the error out that the method produced, could you try to reinstall it (including its dependencies) using: devtools::install_github("dynverse/dynwrap@devel", dependencies=TRUE) ? It is important that also its dependencies are upgraded.

Now if you would rerun a method, you will normally still get an error message, but one produced by the methods themselves. These are sometimes easy to resolve (e.g. a parameter that is outside the allowed range). In other cases, a method errors with reasons beyond our control. This should however not be a big problem for PAGA, slingshot or other top performing methods in our benchmark, given that they error 0-5% of the time on our benchmarking datasets (https://github.com/dynverse/dynbenchmark_results/blob/master/08-summary/results_suppfig.pdf).

Thank you

Wouter

davisidarta commented 5 years ago

Hello there Wouter I've proceeded as you recommended and used devtools::install_github("dynverse/dynwrap@devel", dependencies=TRUE)

I still get an error that doesn't seem to be from my data itself:

> model <- infer_trajectory(dataset, first(methods_selected))
Error traceback:
1: /usr/local/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Traceback (most recent call last):
  File "/code/run.py", line 18, in <module>
    data = h5py.File("/ti/input/data.h5", "r")
  File "/usr/local/lib/python3.6/site-packages/h5py/_hl/files.py", line 312, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
  File "/usr/local/lib/python3.6/site-packages/h5py/_hl/files.py", line 142, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 78, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = '/ti/input/data.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

Error: Error during trajectory inference

What do you guys think of this? Any insights on how to solve it? It would be great to run PAGA in R :)

zouter commented 5 years ago

Hi! Hmmm I wonder whether docker can mount the directories correctly. Can you try to do dynwrap::test_docker_installation(detailed = TRUE) and see whether everything runs fine?

Thanks Wouter

davisidarta commented 5 years ago

Hello Wouter!

I've runned dynwrap::test_docker_installation(detailed = TRUE) and got the following output

Everything seems to be fine with Docker

> dynwrap::test_docker_installation(detailed = TRUE)
<U+2714> Docker is installed
<U+2714> Docker daemon is running
<U+2714> Docker is at correct version (>1.0): 1.39
<U+2714> Docker is in linux mode
<U+2714> Docker can pull images
<U+2714> Docker can run image
<U+2714> Docker can mount temporary volumes
<U+2714> Docker test successful -----------------------------------------------------------------
[1] TRUE

Any other ideias on what could be the issue?

Thanks in advance for your kind attention and support!

Edit: I've tried a clean installation of everything over again, and now I get this error:

model <- infer_trajectory(dataset, first(methods_selected))
Error traceback:
1: Killed

Error: Error during trajectory inference
davisidarta commented 5 years ago

Was anybody able to have any advance on this? After a full reinstallation of my OS (Windows 10), I've got a similar error after running infer_trajectory:

Error traceback:
1: /usr/local/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
Traceback (most recent call last):
  File "/code/run.py", line 18, in <module>
    data = h5py.File("/ti/input/data.h5", "r")
  File "/usr/local/lib/python3.6/site-packages/h5py/_hl/files.py", line 312, in __init__
    fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
  File "/usr/local/lib/python3.6/site-packages/h5py/_hl/files.py", line 142, in make_fid
    fid = h5f.open(name, flags, fapl=fapl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 78, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = '/ti/input/data.h5', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0)

Error: Error during trajectory inference