natverse / fafbseg

Support functions for analysis of Drosophila connectomes especially the FAFB-FlyWire whole brain
https://natverse.org/fafbseg/
GNU General Public License v3.0
6 stars 4 forks source link

use fafbseg in an renv, couldn't find cloud-volume #187

Closed artxz closed 11 months ago

artxz commented 11 months ago

I have fafbseg installed in the system library and it works fine. eg.

id = "720575940620061120"
msh <- read_cloudvolume_meshes(id)

I started a new project with renv, now read_cloudvolume_meshes() returns an error:

Error: Please install the python cloudvolume package:
This should normally work:
fafbseg::simple_python('basic')
For more details see ?simple_python or the cloud-volume docshttps://github.com/seung-lab/cloud-volume#setup
If you have already installed cloudvolume but it is not found
then R probably can't find the relevant version of Python
Do:
usethis::edit_r_environ()
to point to the right python
e.g. RETICULATE_PYTHON="/opt/miniconda3/envs/r-reticulate/bin/python"
12.
stop(call. = F, "Please install the python cloudvolume package:\n", 
"This should normally work:\n", "fafbseg::simple_python('basic')\n", 
"For more details see ?simple_python or the cloud-volume docs", 
"https://github.com/seung-lab/cloud-volume#setup\n", "If you have already installed cloudvolume but it is not found\n", ... at cloudvolume-reticulate.R#7
11.
value[[3L]](cond)
10.
tryCatchOne(expr, names, parentenv, handlers[[1L]])
9.
tryCatchList(expr, classes, parentenv, handlers)
8.
tryCatch(cv <- reticulate::import("cloudvolume"), error = function(e) {
message(as.character(e))
stop(call. = F, "Please install the python cloudvolume package:\n", 
"This should normally work:\n", "fafbseg::simple_python('basic')\n", ... at cloudvolume-reticulate.R#3
7.
(function (min_version = NULL) 
{
check_reticulate()
tryCatch(cv <- reticulate::import("cloudvolume"), error = function(e) { ...
6.
eval(mc, parent.frame())
5.
eval(mc, parent.frame())
4.
withVisible(eval(mc, parent.frame()))
3.
check_cloudvolume_reticulate() at cloudvolume-reticulate.R#68
2.
save_cloudvolume_meshes(segments, savedir = savedir, ..., cloudvolume.url = cloudvolume.url) at cloudvolume-reticulate.R#207
1.
read_cloudvolume_meshes(id)

dr_fafbseg() returns the following (omitting the long fafbseg package options section):

> dr_fafbseg()
R packages
----
fafbseg package:
Version: 0.13
GithubSHA1: 379cb24d0b971392b37bfeb4394b3c94a2ea2567

-- File: 
R arrow package:  13.0.0.1 
FlyWire
----
Checking secrets folder for tokens from R: C:/Users/artxz/.cloudvolume/secrets 
Valid FlyWire ChunkedGraph token is set and found by R!
Please use simple_python to install python+cloudvolume for full access to flywire API!

1 FlyWire/CloudVolume credential files available at
C:/Users/artxz/.cloudvolume/secrets
[1] "cave-secret.json"
Error : invalid version specification ‘NA’

Flywire cloudvolume URL: graphene://https://prod.flywire-daf.com/segmentation/table/fly_v31
Google FFN1 segmentation
----
* Set the `fafbseg.skelziproot` option:
   `options(fafbseg.skelziproot="/path/to/zips")` 
  if you want to use FFN1 skeleton files!

Python
----
python:         C:/Users/artxz/AppData/Local/r-miniconda/envs/r-reticulate/python.exe
libpython:      C:/Users/artxz/AppData/Local/r-miniconda/envs/r-reticulate/python38.dll
pythonhome:     C:/Users/artxz/AppData/Local/r-miniconda/envs/r-reticulate
version:        3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 05:59:00) [MSC v.1929 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:/Users/artxz/AppData/Local/r-miniconda/envs/r-reticulate/Lib/site-packages/numpy
numpy_version:  1.22.3

NOTE: Python version was forced by use_python() function

          package version        requirement channel installed  core
1    cloud-volume   8.5.4 cloud-volume=8.5.4    pypi      TRUE  TRUE
2      caveclient  4.12.5  caveclient=4.12.5    pypi      TRUE  TRUE
3    seatable-api   2.5.5 seatable-api=2.5.5    pypi      TRUE  TRUE
4         pyarrow   7.0.0      pyarrow=7.0.0    pypi      TRUE FALSE
5         fafbseg    <NA>               <NA>    <NA>     FALSE FALSE
6         DracoPy    <NA>               <NA>    <NA>     FALSE FALSE
7       meshparty    <NA>               <NA>    <NA>     FALSE FALSE
8        skeletor    <NA>               <NA>    <NA>     FALSE FALSE
9        pykdtree    <NA>               <NA>    <NA>     FALSE FALSE
10       pyembree    <NA>               <NA>    <NA>     FALSE FALSE
11 pychunkedgraph    <NA>               <NA>    <NA>     FALSE FALSE
12        igneous    <NA>               <NA>    <NA>     FALSE FALSE
13      fastremap  1.12.2   fastremap=1.12.2    pypi      TRUE FALSE
14      ncollpyde    <NA>               <NA>    <NA>     FALSE FALSE

cloud-volume seems available though there is an error at the beginning: Error : invalid version specification ‘NA’.

Any ideas ?

jefferis commented 11 months ago

Dear Arthur, thanks for the report. I think that the error message associated with trying to import cloudvolume in your renv evnvironment may be hidden.

Can you try importing directly

 reticulate::import(“cloudvolume”) 

And reporting any observed error message?

Sent from my iPhone

On 5 Oct 2023, at 21:37, ArthurZhao @.***> wrote:

 I have fafbseg installed in the system library and it works fine. eg.

id = "720575940620061120" msh <- read_cloudvolume_meshes(id) I started a new project with renv, now read_cloudvolume_meshes() returns an error:

Error: Please install the python cloudvolume package: This should normally work: fafbseg::simple_python('basic') For more details see ?simple_python or the cloud-volume docshttps://github.com/seung-lab/cloud-volume#setup If you have already installed cloudvolume but it is not found then R probably can't find the relevant version of Python Do: usethis::edit_r_environ() to point to the right python e.g. RETICULATE_PYTHON="/opt/miniconda3/envs/r-reticulate/bin/python" 12. stop(call. = F, "Please install the python cloudvolume package:\n", "This should normally work:\n", "fafbseg::simple_python('basic')\n", "For more details see ?simple_python or the cloud-volume docs", "https://github.com/seung-lab/cloud-volume#setup\n", "If you have already installed cloudvolume but it is not found\n", ... at cloudvolume-reticulate.R#7 11. value[3L] 10. tryCatchOne(expr, names, parentenv, handlers[[1L]]) 9. tryCatchList(expr, classes, parentenv, handlers) 8. tryCatch(cv <- reticulate::import("cloudvolume"), error = function(e) { message(as.character(e)) stop(call. = F, "Please install the python cloudvolume package:\n", "This should normally work:\n", "fafbseg::simple_python('basic')\n", ... at cloudvolume-reticulate.R#3 7. (function (min_version = NULL) { check_reticulate() tryCatch(cv <- reticulate::import("cloudvolume"), error = function(e) { ... 6. eval(mc, parent.frame()) 5. eval(mc, parent.frame()) 4. withVisible(eval(mc, parent.frame())) 3. check_cloudvolume_reticulate() at cloudvolume-reticulate.R#68 2. save_cloudvolume_meshes(segments, savedir = savedir, ..., cloudvolume.url = cloudvolume.url) at cloudvolume-reticulate.R#207 1. read_cloudvolume_meshes(id) dr_fafbseg() returns the following (omitting the long fafbseg package options section):

dr_fafbseg() R packages

fafbseg package: Version: 0.13 GithubSHA1: 379cb24d0b971392b37bfeb4394b3c94a2ea2567

-- File: R arrow package: 13.0.0.1 FlyWire

Checking secrets folder for tokens from R: C:/Users/artxz/.cloudvolume/secrets Valid FlyWire ChunkedGraph token is set and found by R! Please use simple_python to install python+cloudvolume for full access to flywire API!

1 FlyWire/CloudVolume credential files available at C:/Users/artxz/.cloudvolume/secrets [1] "cave-secret.json" Error : invalid version specification ‘NA’

Flywire cloudvolume URL: graphene://https://prod.flywire-daf.com/segmentation/table/fly_v31 Google FFN1 segmentation

  • Set the fafbseg.skelziproot option: options(fafbseg.skelziproot="/path/to/zips") if you want to use FFN1 skeleton files!

Python

python: C:/Users/artxz/AppData/Local/r-miniconda/envs/r-reticulate/python.exe libpython: C:/Users/artxz/AppData/Local/r-miniconda/envs/r-reticulate/python38.dll pythonhome: C:/Users/artxz/AppData/Local/r-miniconda/envs/r-reticulate version: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 05:59:00) [MSC v.1929 64 bit (AMD64)] Architecture: 64bit numpy: C:/Users/artxz/AppData/Local/r-miniconda/envs/r-reticulate/Lib/site-packages/numpy numpy_version: 1.22.3

NOTE: Python version was forced by use_python() function

      package version        requirement channel installed  core

1 cloud-volume 8.5.4 cloud-volume=8.5.4 pypi TRUE TRUE 2 caveclient 4.12.5 caveclient=4.12.5 pypi TRUE TRUE 3 seatable-api 2.5.5 seatable-api=2.5.5 pypi TRUE TRUE 4 pyarrow 7.0.0 pyarrow=7.0.0 pypi TRUE FALSE 5 fafbseg FALSE FALSE 6 DracoPy FALSE FALSE 7 meshparty FALSE FALSE 8 skeletor FALSE FALSE 9 pykdtree FALSE FALSE 10 pyembree FALSE FALSE 11 pychunkedgraph FALSE FALSE 12 igneous FALSE FALSE 13 fastremap 1.12.2 fastremap=1.12.2 pypi TRUE FALSE 14 ncollpyde FALSE FALSE cloud-volume seems available though there is an error at the beginning: Error : invalid version specification ‘NA’.

Any ideas ?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

artxz commented 11 months ago

Dear Greg, please see the following:

> reticulate::import("cloudvolume")
Error in py_module_import(module, convert = convert) : 
  NameError: name 'base_events' is not defined
Run `reticulate::py_last_error()` for details.
> reticulate::py_last_error()

-- Python Exception Message -------------------------------------------
Traceback (most recent call last):
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 119, in _find_and_load_hook
    return _run_hook(name, _hook)
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 93, in _run_hook
    module = hook()
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 117, in _hook
    return _find_and_load(name, import_)
  File "C:\Users\artxz\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\cloudvolume\__init__.py", line 49, in <module>
    from .cloudvolume import CloudVolume, register_plugin
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 119, in _find_and_load_hook
    return _run_hook(name, _hook)
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 93, in _run_hook
    module = hook()
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 117, in _hook
    return _find_and_load(name, import_)
  File "C:\Users\artxz\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\cloudvolume\cloudvolume.py", line 10, in <module>
    from .paths import strict_extract, to_https_protocol
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 119, in _find_and_load_hook
    return _run_hook(name, _hook)
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 93, in _run_hook
    module = hook()
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 117, in _hook
    return _find_and_load(name, import_)
  File "C:\Users\artxz\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\cloudvolume\paths.py", line 1, in <module>
    import cloudfiles.paths
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 119, in _find_and_load_hook
    return _run_hook(name, _hook)
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 93, in _run_hook
    module = hook()
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 117, in _hook
    return _find_and_load(name, import_)
  File "C:\Users\artxz\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\cloudfiles\__init__.py", line 12, in <module>
    from .cloudfiles import CloudFile, CloudFiles, dl
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 119, in _find_and_load_hook
    return _run_hook(name, _hook)
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 93, in _run_hook
    module = hook()
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 117, in _hook
    return _find_and_load(name, import_)
  File "C:\Users\artxz\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\cloudfiles\cloudfiles.py", line 42, in <module>
    from .interfaces import (
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 119, in _find_and_load_hook
    return _run_hook(name, _hook)
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 93, in _run_hook
    module = hook()
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 117, in _hook
    return _find_and_load(name, import_)
  File "C:\Users\artxz\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\cloudfiles\interfaces.py", line 16, in <module>
    import tenacity
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 119, in _find_and_load_hook
    return _run_hook(name, _hook)
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 93, in _run_hook
    module = hook()
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 117, in _hook
    return _find_and_load(name, import_)
  File "C:\Users\artxz\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tenacity\__init__.py", line 512, in <module>
    from tenacity._asyncio import AsyncRetrying  # noqa:E402,I100
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 119, in _find_and_load_hook
    return _run_hook(name, _hook)
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 93, in _run_hook
    module = hook()
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 117, in _hook
    return _find_and_load(name, import_)
  File "C:\Users\artxz\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tenacity\_asyncio.py", line 21, in <module>
    from asyncio import sleep
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 119, in _find_and_load_hook
    return _run_hook(name, _hook)
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 93, in _run_hook
    module = hook()
  File "C:\Users\artxz\Dropbox (HHMI)\sync_userA\Documents\ReiserGroup\p_sexdim\renv\library\R-4.1\x86_64-w64-mingw32\reticulate\python\rpytools\loader.py", line 117, in _hook
    return _find_and_load(name, import_)
  File "C:\Users\artxz\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\asyncio\__init__.py", line 26, in <module>
    __all__ = (base_events.__all__ +
NameError: name 'base_events' is not defined

-- R Traceback --------------------------------------------------------
    x
 1. \-reticulate::import("cloudvolume")
 2.   \-reticulate:::py_module_import(module, convert = convert)
jefferis commented 11 months ago

Ah. Can you try updating to the development version of reticulate from GitHub? I think there is a bug related to this. You’ll need to restart R afterwards of course.

artxz commented 11 months ago

Installing "rstudio/reticulate" from GitHub solves the problem. Thanks again, Greg!