search

erikjskie / ensemblemerge

0 stars 0 forks source link

Install error on colab #8

Closed gongx030 closed 3 years ago

gongx030 commented 3 years ago

There is an error when installing ensemblemerge on colab. The error message is like this:

Downloading GitHub repo erikjskie/ensemblemerge@HEAD

Skipping 2 packages not available: SingleCellExperiment, SummarizedExperiment

✔ checking for file ‘/tmp/RtmpVVWkxB/remotes4832115de9/erikjskie-ensemblemerge-a7a71a0efb2ce439c486fa7962fb3dfc6db7fc22/DESCRIPTION’ ─ preparing ‘ensemblemerge’: ✔ checking DESCRIPTION meta-information ─ checking for LF line-endings in source and make files and shell scripts ─ checking for empty or unneeded directories ─ building ‘ensemblemerge_0.0.0.9000.tar.gz’

Installing package into ‘/usr/local/lib/R/site-library’ (as ‘lib’ is unspecified)

Error: Failed to install 'ensemblemerge' from GitHub: (converted from warning) installation of package ‘/tmp/RtmpVVWkxB/file48627fd3b7/ensemblemerge_0.0.0.9000.tar.gz’ had non-zero exit status Traceback:

  1. devtools::install_github("erikjskie/ensemblemerge", auth_token = "ce7a09a6a501d67660a59724a931805364ec6f5e")
  2. pkgbuild::with_build_tools({ . ellipsis::check_dots_used(action = getOption("devtools.ellipsis_action", . rlang::warn)) . { . remotes <- lapply(repo, github_remote, ref = ref, subdir = subdir, . auth_token = auth_token, host = host) . install_remotes(remotes, auth_token = auth_token, host = host, . dependencies = dependencies, upgrade = upgrade, force = force, . quiet = quiet, build = build, build_opts = build_opts, . build_manual = build_manual, build_vignettes = build_vignettes, . repos = repos, type = type, ...) . } . }, required = FALSE)
  3. install_remotes(remotes, auth_token = auth_token, host = host, . dependencies = dependencies, upgrade = upgrade, force = force, . quiet = quiet, build = build, build_opts = build_opts, build_manual = build_manual, . build_vignettes = build_vignettes, repos = repos, type = type, . ...)
  4. tryCatch(res[[i]] <- install_remote(remotes[[i]], ...), error = function(e) { . stop(remote_install_error(remotes[[i]], e)) . })
  5. tryCatchList(expr, classes, parentenv, handlers)
  6. tryCatchOne(expr, names, parentenv, handlers[[1L]])
  7. value[3L]
gongx030 commented 3 years ago

When installing ensemblmerge on MSI, I also got error. The error message on MSI appears to be more detailed. It appears that the error is related to fbpca.

devtools::install_github("erikjskie/ensemblemerge") Downloading GitHub repo erikjskie/ensemblemerge@HEAD These packages have more recent versions available. It is recommended to update all of them. Which would you like to update?

1: All 2: CRAN packages only 3: None 4: Rcpp (1.0.5 -> 1.0.6 ) [CRAN] 5: utf8 (1.1.4 -> 1.2.1 ) [CRAN] 6: crayon (1.3.4 -> 1.4.1 ) [CRAN] 7: cli (2.2.0 -> 2.3.1 ) [CRAN] 8: mime (0.9 -> 0.10 ) [CRAN] 9: vctrs (0.3.5 -> 0.3.7 ) [CRAN] 10: pillar (1.4.7 -> 1.5.1 ) [CRAN] 11: fansi (0.4.1 -> 0.4.2 ) [CRAN] 12: lifecycle (0.2.0 -> 1.0.0 ) [CRAN] 13: farver (2.0.3 -> 2.1.0 ) [CRAN] 14: xfun (0.19 -> 0.22 ) [CRAN] 15: knitr (1.30 -> 1.31 ) [CRAN] 16: withr (2.3.0 -> 2.4.1 ) [CRAN] 17: tibble (3.0.4 -> 3.1.0 ) [CRAN] 18: rlang (0.4.9 -> 0.4.10 ) [CRAN] 19: isoband (0.2.3 -> 0.2.4 ) [CRAN] 20: ggplot2 (3.3.2 -> 3.3.3 ) [CRAN] 21: RcppEigen (0.3.3.7.0 -> 0.3.3.9.1 ) [CRAN] 22: spatstat.... (2.0-0 -> 2.1-0 ) [CRAN] 23: spatstat.... (2.0-0 -> 2.1-0 ) [CRAN] 24: spatstat.... (1.2-1 -> 2.0-0 ) [CRAN] 25: spatstat.... (1.65-5 -> 2.0-1 ) [CRAN] 26: rappdirs (0.3.1 -> 0.3.3 ) [CRAN] 27: promises (1.1.1 -> 1.2.0.1 ) [CRAN] 28: parallelly (1.23.0 -> 1.24.0 ) [CRAN] 29: RcppArmad... (0.10.1.2.2 -> 0.10.2.2.0) [CRAN] 30: matrixStats (0.57.0 -> 0.58.0 ) [CRAN] 31: caTools (1.18.1 -> 1.18.2 ) [CRAN] 32: cpp11 (0.2.4 -> 0.2.7 ) [CRAN] 33: crosstalk (1.1.0.1 -> 1.1.1 ) [CRAN] 34: dplyr (1.0.2 -> 1.0.5 ) [CRAN] 35: tidyr (1.1.2 -> 1.1.3 ) [CRAN] 36: zoo (1.8-8 -> 1.8-9 ) [CRAN] 37: psych (2.0.12 -> 2.1.3 ) [CRAN] 38: Hmisc (4.4-2 -> 4.5-0 ) [CRAN] 39: spatstat.... (1.65-5 -> 2.0-0 ) [CRAN] 40: Seurat (4.0.0 -> 4.0.1 ) [CRAN]

Enter one or more numbers, or an empty line to skip updates: ✔ checking for file ‘/panfs/roc/scratch/gongx030/Rtmpy55xuS/remotes30aab3774697/erikjskie-ensemblemerge-a7a71a0efb2ce439c486fa7962fb3dfc6db7fc22/DESCRIPTION’ (427ms) ─ preparing ‘ensemblemerge’: (552ms) ✔ checking DESCRIPTION meta-information ─ checking for LF line-endings in source and make files and shell scripts (937ms) ─ checking for empty or unneeded directories ─ building ‘ensemblemerge_0.0.0.9000.tar.gz’ Warning: invalid uid value replaced by that for user 'nobody'

Detailed traceback: File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/R/library/reticulate/python/rpytools/loader.py", line 24, in _import_hook level=level File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/python3.7/site-packages/scanorama/init.py", line 1, in from .scanorama import * File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/R/library/reticulate/python/rpytools/loader.py", line 24, in _import_hook level=level File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/python3.7/site-packages/scanorama/scanorama.py", line 16, in from .utils import plt, dispersion, reduce_dimensionality File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/R/library/reticulate/python/rpytools/loader.py", line 24, in _import_hook level=level File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/python3.7/site-packages/scanorama/utils.py", line 2, in from fbpca import pca

Error: loading failed Execution halted ERROR: loading failed

erikjskie commented 3 years ago

Thanks i'll take a look

erikjskie commented 3 years ago

i'm loading up a testing environment now to see, but it looks like fbpca might not be installed? try "system("pip3 install fbpca")"

gongx030 commented 3 years ago

i'm loading up a testing environment now to see, but it looks like fbpca might not be installed? try "system("pip3 install fbpca")"

I did that on MSI but I still got similar error. See below:

system('pip3 install fbpca') Requirement already satisfied: fbpca in /panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/python3.7/site-packages (1.0) devtools::install_github("erikjskie/ensemblemerge", auth_token = "ce7a09a6a501d67660a59724a931805364ec6f5e") Downloading GitHub repo erikjskie/ensemblemerge@HEAD These packages have more recent versions available. It is recommended to update all of them. Which would you like to update?

1: All 2: CRAN packages only 3: None 4: Rcpp (1.0.5 -> 1.0.6 ) [CRAN] 5: utf8 (1.1.4 -> 1.2.1 ) [CRAN] 6: crayon (1.3.4 -> 1.4.1 ) [CRAN] 7: cli (2.2.0 -> 2.3.1 ) [CRAN] 8: mime (0.9 -> 0.10 ) [CRAN] 9: vctrs (0.3.5 -> 0.3.7 ) [CRAN] 10: pillar (1.4.7 -> 1.5.1 ) [CRAN] 11: fansi (0.4.1 -> 0.4.2 ) [CRAN] 12: lifecycle (0.2.0 -> 1.0.0 ) [CRAN] 13: farver (2.0.3 -> 2.1.0 ) [CRAN] 14: xfun (0.19 -> 0.22 ) [CRAN] 15: knitr (1.30 -> 1.31 ) [CRAN] 16: withr (2.3.0 -> 2.4.1 ) [CRAN] 17: tibble (3.0.4 -> 3.1.0 ) [CRAN] 18: rlang (0.4.9 -> 0.4.10 ) [CRAN] 19: isoband (0.2.3 -> 0.2.4 ) [CRAN] 20: ggplot2 (3.3.2 -> 3.3.3 ) [CRAN] 21: RcppEigen (0.3.3.7.0 -> 0.3.3.9.1 ) [CRAN] 22: spatstat.... (2.0-0 -> 2.1-0 ) [CRAN] 23: spatstat.... (2.0-0 -> 2.1-0 ) [CRAN] 24: spatstat.... (1.2-1 -> 2.0-0 ) [CRAN] 25: spatstat.... (1.65-5 -> 2.0-1 ) [CRAN] 26: rappdirs (0.3.1 -> 0.3.3 ) [CRAN] 27: promises (1.1.1 -> 1.2.0.1 ) [CRAN] 28: parallelly (1.23.0 -> 1.24.0 ) [CRAN] 29: RcppArmad... (0.10.1.2.2 -> 0.10.2.2.0) [CRAN] 30: matrixStats (0.57.0 -> 0.58.0 ) [CRAN] 31: caTools (1.18.1 -> 1.18.2 ) [CRAN] 32: cpp11 (0.2.4 -> 0.2.7 ) [CRAN] 33: crosstalk (1.1.0.1 -> 1.1.1 ) [CRAN] 34: dplyr (1.0.2 -> 1.0.5 ) [CRAN] 35: tidyr (1.1.2 -> 1.1.3 ) [CRAN] 36: zoo (1.8-8 -> 1.8-9 ) [CRAN] 37: psych (2.0.12 -> 2.1.3 ) [CRAN] 38: Hmisc (4.4-2 -> 4.5-0 ) [CRAN] 39: spatstat.... (1.65-5 -> 2.0-0 ) [CRAN] 40: Seurat (4.0.0 -> 4.0.1 ) [CRAN]

Enter one or more numbers, or an empty line to skip updates: ✔ checking for file ‘/panfs/roc/scratch/gongx030/Rtmpy55xuS/remotes30aab3774697/erikjskie-ensemblemerge-a7a71a0efb2ce439c486fa7962fb3dfc6db7fc22/DESCRIPTION’ (427ms) ─ preparing ‘ensemblemerge’: (552ms) ✔ checking DESCRIPTION meta-information ─ checking for LF line-endings in source and make files and shell scripts (937ms) ─ checking for empty or unneeded directories ─ building ‘ensemblemerge_0.0.0.9000.tar.gz’ Warning: invalid uid value replaced by that for user 'nobody'

Detailed traceback: File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/R/library/reticulate/python/rpytools/loader.py", line 24, in _import_hook level=level File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/python3.7/site-packages/scanorama/init.py", line 1, in from .scanorama import * File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/R/library/reticulate/python/rpytools/loader.py", line 24, in _import_hook level=level File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/python3.7/site-packages/scanorama/scanorama.py", line 16, in from .utils import plt, dispersion, reduce_dimensionality File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/R/library/reticulate/python/rpytools/loader.py", line 24, in _import_hook level=level File "/panfs/roc/groups/10/garrydj/gongx030/.conda/envs/r403/lib/python3.7/site-packages/scanorama/utils.py", line 2, in from fbpca import pca

Error: loading failed Execution halted ERROR: loading failed

gongx030 commented 3 years ago

In Python (MSI) I got something like this:

>>> from fbpca import pca
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'pca' from 'fbpca' (/home/garrydj/gongx030/src/cellranger-3.0.2/cellranger-cs/3.0.2/lib/python/fbpca/__init__.py)

I am using the fbpca version 1.0:

> (r403) gongx030@cn3002 [~/rlib/analysis/scco_project] % conda list fbpca
> # packages in environment at /home/garrydj/gongx030/.conda/envs/r403:
> #
> # Name                    Version                   Build  Channel
> fbpca                     1.0                       <pip>
>
gongx030 commented 3 years ago

Please fix this issue ASAP.

erikjskie commented 3 years ago

I tried recreating the bug in colab and MSI (both under my normal scRNAseq environment and a fresh conda environment) but I wasn't getting issues...

I pushed a potential fix to main where I change the initialization of the reticulate environment until it's called by a method that uses it so python packages won't interfere with the install. Let me know if that fixes the problem.

Another bug i'm working through is Seurat must have updated because it's as.Seurat() function that converts SingleCellExperiment objects to Seurat objects isn't working on SingleCellExperiment objects anymore (just happened today) i'm testing earlier versions now to confirm and i'll just enforce install of a specific Seurat version if that is the case but I just wanted to make sure your aware when your setting up environments.

gongx030 commented 3 years ago

The loading of ensemblmerge on colab failed again. Here is the notebook (https://github.com/gongx030/scco_project/blob/main/notebooks/colab_loading_test.ipynb or https://colab.research.google.com/github/gongx030/scco_project/blob/main/notebooks/colab_loading_test.ipynb) for reproducing the error. Before you push the update to the master branch, be sure to test it on the above notebook. This type of error should not happen again. Thanks.

erikjskie commented 3 years ago

I am loading your notebook now for testing but I reverted the changes I made yesterday so it should work for you now.

gongx030 commented 3 years ago

I am loading your notebook now for testing but I reverted the changes I made yesterday so it should work for you now.

Works right now. Thanks.

gongx030 commented 3 years ago

I made a change in colab_loading_test.ipynb to install an additional python package tensorflow_addons. See https://colab.research.google.com/github/gongx030/scco_project/blob/main/notebooks/colab_loading_test.ipynb#scrollTo=QCF4C9wQ2v3y&line=2&uniqifier=1