TooManyCellsR

[pedriniedoardo]

Hello, I am trying to run the example code for tooManyCells function from the TooManyCellsR package in R. I am using windows 10. I have followed the docker workflow. I can confirm the container works:

PS C:\Windows\system32> docker run -it --rm -v "/home/username:/home/username" gregoryschwartz/too-many-cells:0.1.5.0 -h
too-many-cells, Gregory W. Schwartz. Clusters and analyzes single cell data.

Usage: too-many-cells (make-tree | interactive | differential | diversity |
                      paths)

Available options:
  -h,--help                Show this help text

Available commands:
  make-tree
  interactive
  differential
  diversity
  paths

In R I follow the example code from the TooManyCells function and specify the docker argument

library(TooManyCellsR)
input <- system.file("extdata", "mat.csv", package="TooManyCellsR")
inputLabels <- system.file("extdata", "labels.csv", package="TooManyCellsR")
df = read.csv(input, row.names = 1, header = TRUE)
mat = Matrix::Matrix(as.matrix(df), sparse = TRUE)
labelsDf = read.csv(inputLabels, header = TRUE)
res = tooManyCells(docker = "gregoryschwartz/too-many-cells:0.1.5.0",mat, labels = labelsDf
                    , args = c( "make-tree"
                                , "--no-filter"
                                , "--normalization", "NoneNorm"
                                , "--draw-max-node-size", "40"
                                , "--draw-max-leaf-node-size", "70"
                    )
)

I can create all the object but res, with the following error:

Error in wrap.url(file, load.image.internal) : File not found
In addition: Warning messages:
1: In dir.create(output, recursive = TRUE) : 'out' already exists
2: In system2("docker", args = c(dockerArgs, args, autoArgs), stdout = TRUE) :
  running command '"docker" run -i --rm -v C:\Users\pedri\AppData\Local\Temp\Rtmp8sTiO7:C:\Users\pedri\AppData\Local\Temp\Rtmp8sTiO7 -v C:\Users\pedri\Documents\out:C:\Users\pedri\Documents\out gregoryschwartz/too-many-cells:0.1.5.0 make-tree --no-filter --normalization NoneNorm --draw-max-node-size 40 --draw-max-leaf-node-size 70 --matrix-path C:\Users\pedri\AppData\Local\Temp\Rtmp8sTiO7 --output C:\Users\pedri\Documents\out --labels-file C:\Users\pedri\AppData\Local\Temp\Rtmp8sTiO7/labels.csv' had status 125

any idea how to fix the issue?

here my session:

> sessionInfo()
R version 3.6.2 (2019-12-12)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18362)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] TooManyCellsR_0.1.1.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.3       lattice_0.20-38  png_0.1-7        tiff_0.1-5       grid_3.6.2       imager_0.42.1    magrittr_1.5     stringi_1.4.3   
 [9] rlang_0.4.2      Matrix_1.2-18    bmp_0.3          tools_3.6.2      stringr_1.4.0    purrr_0.3.3      igraph_1.2.4.2   jpeg_0.1-8.1    
[17] compiler_3.6.2   pkgconfig_2.0.3  readbitmap_0.1.5

[GregorySchwartz]

I unfortunately don't have Windows to test the fix on, but try the newest version of TooManyCellsR on github. I believe it was a filepath issue with Windows which should be fixed.

[pedriniedoardo]

Hello @GregorySchwartz , thank you for looking into that.

I tried what you suggested but I still got the same error

> res <- tooManyCells(docker = "gregoryschwartz/too-many-cells:0.2.2.0",
+                    mat,
+                    labels = labelsDf,
+                    args = c( "make-tree"
+                                 , "--no-filter"
+                                 , "--normalization", "NoneNorm"
+                                 , "--draw-max-node-size", "40"
+                                 , "--draw-max-leaf-node-size", "70"
+                     )
+ )
Error in wrap.url(file, load.image.internal) : File not found
In addition: Warning messages:
1: In dir.create(output, recursive = TRUE) : 'out' already exists
2: In system2("docker", args = c(dockerArgs, args, autoArgs), stdout = TRUE) :
  running command '"docker" run -i --rm -v C:\Users\pedri\AppData\Local\Temp\RtmpQVhJ7w:C:\Users\pedri\AppData\Local\Temp\RtmpQVhJ7w -v C:\Users\pedri\Documents\out:C:\Users\pedri\Documents\out gregoryschwartz/too-many-cells:0.2.2.0 make-tree --no-filter --normalization NoneNorm --draw-max-node-size 40 --draw-max-leaf-node-size 70 --matrix-path C:\Users\pedri\AppData\Local\Temp\RtmpQVhJ7w --output C:\Users\pedri\Documents\out --labels-file C:\Users\pedri\AppData\Local\Temp\RtmpQVhJ7w/labels.csv' had status 125

here to confirm the effective new version of the R package

> sessionInfo()
R version 3.6.2 (2019-12-12)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18362)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] TooManyCellsR_0.1.1.1

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.3        compiler_3.6.2    imager_0.42.1     prettyunits_1.0.2 remotes_2.1.0     tools_3.6.2       testthat_2.3.1   
 [8] digest_0.6.25     pkgbuild_1.0.6    pkgload_1.0.2     memoise_1.1.0     lattice_0.20-38   pkgconfig_2.0.3   png_0.1-7        
[15] rlang_0.4.5       Matrix_1.2-18     igraph_1.2.4.2    cli_2.0.2         rstudioapi_0.10   curl_4.3          bmp_0.3          
[22] readbitmap_0.1.5  stringr_1.4.0     withr_2.1.2       desc_1.2.0        fs_1.3.1          devtools_2.2.1    rprojroot_1.3-2  
[29] grid_3.6.2        glue_1.3.1        R6_2.4.1          jpeg_0.1-8.1      processx_3.4.1    fansi_0.4.1       sessioninfo_1.1.1
[36] purrr_0.3.3       callr_3.4.0       magrittr_1.5      backports_1.1.5   ps_1.3.0          ellipsis_0.3.0    usethis_1.5.1    
[43] assertthat_0.2.1  tiff_0.1-5        stringi_1.4.6     crayon_1.3.4   

I also pulled the newer docker version but doesn't seem to fix the error

PS C:\Windows\system32> docker run -it --rm -v "/home/username:/home/username" gregoryschwartz/too-many-cells:0.2.2.0 -h
too-many-cells, Gregory W. Schwartz. Clusters and analyzes single cell data.

Usage: too-many-cells (make-tree | interactive | differential | diversity |
                      paths)

Available options:
  -h,--help                Show this help text

Available commands:
  make-tree
  interactive
  differential
  diversity
  paths

[GregorySchwartz]

Did you increase the memory usable by docker (https://docs.docker.com/config/containers/resource_constraints/)? It's possible docker ran out of memory and crashed silently, not producing output.

[pedriniedoardo]

just tried to increase the specs of docker, still get the same error.

PS C:\Windows\system32> docker info
Client:
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 19.03.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.19.76-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 5.809GiB
 Name: docker-desktop
 ID: JZHY:T7HC:FIAQ:LDD2:K7Z5:2OJX:WMPU:PGF3:OKIK:T5S5:5H3K:4HQ7
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 33
  Goroutines: 50
  System Time: 2020-03-03T18:00:48.121623967Z
  EventsListeners: 3
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

considering the size the sample data I assume it should be more than enough?!

[GregorySchwartz]

Are you able to run too-many-cells without the wrapper?

docker run -i --rm -v C:\Users\pedri:C:\Users\pedri gregoryschwartz/too-many-cells:0.2.2.0 make-tree --no-filter --normalization NoneNorm --draw-max-node-size 40 --draw-max-leaf-node-size 70 --matrix-path C:\Users\pedri\path\to\mat --output out --labels-file C:\Users\pedri\path\to\labels.csv

[GregorySchwartz]

I wonder if it has to do with the different notation on Windows. Try:

docker run -i --rm -v C:\Users\pedri\path\to\mat.csv:/mat.csv -v C:\Users\pedri\path\to\labels.csv:/labels.csv -v C:\Users\pedri\path\to\out:/out gregoryschwartz/too-many-cells:0.2.2.0 make-tree --no-filter --normalization NoneNorm --draw-max-node-size 40 --draw-max-leaf-node-size 70 --matrix-path /mat.csv --output /out --labels-file /labels.csv

[GregorySchwartz]

If that works, then it's because of the different filesystem of the docker.

[pedriniedoardo]

I guess there is some issue with the windows path, here is the output of the first command

PS C:\Windows\system32> docker run -i --rm -v C:\Users\pedri:C:\Users\pedri gregoryschwartz/too-many-cells:0.2.2.0 make-tree --no-filter --normalization NoneNorm --draw-max-node-size 40 --draw-max-leaf-node-size 70 --matrix-path C:\Users\pedri\Documents\training\bioinformatic\TooManyCells\test\mat.csv --output C:\Users\pedri\Documents\training\bioinformatic\TooManyCells\test --labels-file C:\Users\pedri\Documents\training\bioinformatic\TooManyCells\test\label.csv
C:\Program Files\Docker\Docker\resources\bin\docker.exe: Error response from daemon: invalid mode: \Users\pedri.
See 'C:\Program Files\Docker\Docker\resources\bin\docker.exe run --help'.

the second suggestion start the process but stop after 10%

PS C:\Windows\system32> docker run -i --rm -v C:\Users\pedri\Documents\training\bioinformatic\TooManyCells\test\mat.csv:/mat.csv -v C:\Users\pedri\Documents\training\bioinformatic\TooManyCells\test\label.csv:/labels.csv -v C:\Users\pedri\Documents\training\bioinformatic\TooManyCells\test:/out gregoryschwartz/too-many-cells:0.2.2.0 make-tree --no-filter --normalization NoneNorm --draw-max-node-size 40 --draw-max-leaf-node-size 70 --matrix-path /mat.csv --output /out --labels-file /labels.csv
Planning leaf colors [=====>..............................................]  10%too-many-cells: /labels.csv: openBinaryFile: inappropriate type (is a directory)

[GregorySchwartz]

Try:

docker run -i --rm -v C:\Users\pedri\Documents\training\bioinformatic\TooManyCells\test:/ gregoryschwartz/too-many-cells:0.2.2.0 make-tree --no-filter --normalization NoneNorm --draw-max-node-size 40 --draw-max-leaf-node-size 70 --matrix-path /mat.csv --output /out --labels-file /labels.csv

[pedriniedoardo]

still some error

PS C:\Windows\system32> docker run -i --rm -v C:\Users\pedri\Documents\training\bioinformatic\TooManyCells\test:/ gregoryschwartz/too-many-cells:0.2.2.0 make-tree --no-filter --normalization NoneNorm --draw-max-node-size 40 --draw-max-leaf-node-size 70 --matrix-path /mat.csv --output /out --labels-file /labels.csv
C:\Program Files\Docker\Docker\resources\bin\docker.exe: Error response from daemon: invalid volume specification: '/host_mnt/c/Users/pedri/Documents/training/bioinformatic/TooManyCells/test:/': invalid mount config for type "bind": invalid specification: destination can't be '/'.
See 'C:\Program Files\Docker\Docker\resources\bin\docker.exe run --help'.

[GregorySchwartz]

Try a different mount point:

docker run -i --rm -v C:\Users\pedri\Documents\training\bioinformatic\TooManyCells\test:/work gregoryschwartz/too-many-cells:0.2.2.0 make-tree --no-filter --normalization NoneNorm --draw-max-node-size 40 --draw-max-leaf-node-size 70 --matrix-path /work/mat.csv --output /work/out --labels-file /work/labels.csv

[pedriniedoardo]

yay!

PS C:\Windows\system32> docker run -i --rm -v C:\Users\pedri\Documents\training\bioinformatic\TooManyCells\test:/work gregoryschwartz/too-many-cells:0.2.2.0 make-tree --no-filter --normalization NoneNorm --draw-max-node-size 40 --draw-max-leaf-node-size 70 --matrix-path /work/mat.csv --output /work/out --labels-file /work/labels.csv
Painting sketches [======================================>................]  70%
No labels for leaves based on chosen clumpiness method, skipping clumpiness plot.
Packing up [==============================================================] 100%cell,cluster,path
cell4,1,1/0
cell5,1,1/0
cell1,2,2/0
cell2,2,2/0
cell3,2,2/0
cell6,2,2/0

[GregorySchwartz]

Well the good news is that you have a way to run it, but the bad news is that work should be done on TooManyCellsR before it can be used on Windows due to the disparity between Windows and the docker system.

[pedriniedoardo]

thank you very much @GregorySchwartz ! I will tell people in the lab that in case they want to run your algorithm in R they would have to go with linux (where it works perfectly). if they want to keep using win they would need to work in docker. thank you very much for the support.