search

maximilianh / cellBrowser

main repo: https://github.com/ucscGenomeBrowser/cellBrowser/ - Python pipeline and Javascript scatter plot library for single-cell datasets, http://cellbrowser.rtfd.org
https://github.com/ucscGenomeBrowser/cellBrowser/
GNU General Public License v3.0
104 stars 41 forks source link

ImportError: cannot import name 'cbImportSeurat2Cli' occurs calling cbImportSeurat from command-line on conda install #233

Closed malcook closed 2 years ago

malcook commented 2 years ago 
Traceback (most recent call last):
  File "/n/projects/mec/local/anaconda/envs/ucsc-cell-browser/bin/cbImportSeurat", line 6, in <module>
    from cellbrowser.seurat import cbImportSeurat2Cli
ImportError: cannot import name 'cbImportSeurat2Cli' from 'cellbrowser.seurat' (/n/projects/mec/local/anaconda/envs/ucsc-cell-browser/lib/python3.9/site-packages/cellbrowser/seurat.py)

is occurring whether calling cbImportSeurat with or without command-line options

ucsc-cell-browser was installed on CentOS7 into

conda --version
conda 4.10.3

using 

conda install -c bioconda ucsc-cell-browser -n ucsc-cell-browser -m`

resulting in

conda list
# packages in environment at /n/projects/mec/local/anaconda/envs/ucsc-cell-browser:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             4.5                       1_gnu  
anndata                   0.6.22.post1               py_0    bioconda
blas                      1.0                         mkl  
bottleneck                1.3.2            py39hdd57654_1  
ca-certificates           2021.10.26           h06a4308_2  
certifi                   2021.10.8        py39h06a4308_0  
h5py                      3.6.0            py39ha0f2276_0  
hdf5                      1.10.6               hb1b8bf9_0  
intel-openmp              2021.4.0          h06a4308_3561  
ld_impl_linux-64          2.35.1               h7274673_9  
libffi                    3.3                  he6710b0_2  
libgcc-ng                 9.3.0               h5101ec6_17  
libgfortran-ng            7.5.0               ha8ba4b0_17  
libgfortran4              7.5.0               ha8ba4b0_17  
libgomp                   9.3.0               h5101ec6_17  
libstdcxx-ng              9.3.0               hd4cf53a_17  
mkl                       2021.4.0           h06a4308_640  
mkl-service               2.4.0            py39h7f8727e_0  
mkl_fft                   1.3.1            py39hd3c417c_0  
mkl_random                1.2.2            py39h51133e4_0  
natsort                   7.1.1              pyhd3eb1b0_0  
ncurses                   6.3                  h7f8727e_2  
numexpr                   2.7.3            py39h22e1b3c_1  
numpy                     1.21.2           py39h20f2e39_0  
numpy-base                1.21.2           py39h79a1101_0  
openssl                   1.1.1l               h7f8727e_0  
pandas                    1.3.4            py39h8c16a72_0  
pip                       21.2.4           py39h06a4308_0  
python                    3.9.7                h12debd9_1  
python-dateutil           2.8.2              pyhd3eb1b0_0  
pytz                      2021.3             pyhd3eb1b0_0  
readline                  8.1                  h27cfd23_0  
scipy                     1.7.1            py39h292c36d_2  
setuptools                58.0.4           py39h06a4308_0  
six                       1.16.0             pyhd3eb1b0_0  
sqlite                    3.36.0               hc218d9a_0  
tk                        8.6.11               h1ccaba5_0  
tzdata                    2021e                hda174b7_0  
ucsc-cell-browser         1.1.1              pyhdfd78af_0    bioconda
wheel                     0.37.0             pyhd3eb1b0_1  
xz                        5.2.5                h7b6447c_0  
zlib                      1.2.11               h7f8727e_4

in case it helps:

conda info -a

     active environment : ucsc-cell-browser
    active env location : /n/projects/mec/local/anaconda/envs/ucsc-cell-browser
            shell level : 1
       user config file : /home/mec/.condarc
 populated config files : /n/projects/mec/local/anaconda/.condarc
                          /home/mec/.condarc
          conda version : 4.10.3
    conda-build version : 3.21.5
         python version : 3.8.11.final.0
       virtual packages : __linux=3.10.0=0
                          __glibc=2.17=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /n/projects/mec/local/anaconda  (writable)
      conda av data dir : /n/projects/mec/local/anaconda/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /n/projects/mec/local/anaconda/pkgs
                          /home/mec/.conda/pkgs
       envs directories : /n/projects/mec/local/anaconda/envs
                          /home/mec/.conda/envs
               platform : linux-64
             user-agent : conda/4.10.3 requests/2.26.0 CPython/3.8.11 Linux/3.10.0-862.14.4.el7.x86_64 centos/7.2.1511 glibc/2.17
                UID:GID : 1232:20011
             netrc file : /home/mec/.netrc
           offline mode : False

# conda environments:
#
base                     /n/projects/mec/local/anaconda
02_Scanpy                /n/projects/mec/local/anaconda/envs/02_Scanpy
02_Scanpy_b              /n/projects/mec/local/anaconda/envs/02_Scanpy_b
02_Scanpy`               /n/projects/mec/local/anaconda/envs/02_Scanpy`
datasci_01               /n/projects/mec/local/anaconda/envs/datasci_01
fresh                    /n/projects/mec/local/anaconda/envs/fresh
jupyterhub               /n/projects/mec/local/anaconda/envs/jupyterhub
new2_scanpy              /n/projects/mec/local/anaconda/envs/new2_scanpy
new3_scanpy              /n/projects/mec/local/anaconda/envs/new3_scanpy
new_scanpy               /n/projects/mec/local/anaconda/envs/new_scanpy
r-reticulate             /n/projects/mec/local/anaconda/envs/r-reticulate
                         /n/projects/mec/local/anaconda/envs/r-reticulate/envs/r-reticulate
stats                    /n/projects/mec/local/anaconda/envs/stats
ucsc-cell-browser     *  /n/projects/mec/local/anaconda/envs/ucsc-cell-browser

sys.version: 3.8.11 (default, Aug  3 2021, 15:09:35) ...
sys.prefix: /n/projects/mec/local/anaconda
sys.executable: /n/projects/mec/local/anaconda/bin/python
conda location: /n/projects/mec/local/anaconda/lib/python3.8/site-packages/conda
conda-build: /n/projects/mec/local/anaconda/bin/conda-build
conda-content-trust: /n/projects/mec/local/anaconda/bin/conda-content-trust
conda-convert: /n/projects/mec/local/anaconda/bin/conda-convert
conda-debug: /n/projects/mec/local/anaconda/bin/conda-debug
conda-develop: /n/projects/mec/local/anaconda/bin/conda-develop
conda-env: /n/projects/mec/local/anaconda/bin/conda-env
conda-index: /n/projects/mec/local/anaconda/bin/conda-index
conda-inspect: /n/projects/mec/local/anaconda/bin/conda-inspect
conda-metapackage: /n/projects/mec/local/anaconda/bin/conda-metapackage
conda-pack: /n/projects/mec/local/anaconda/bin/conda-pack
conda-render: /n/projects/mec/local/anaconda/bin/conda-render
conda-repo: /n/projects/mec/local/anaconda/bin/conda-repo
conda-server: /n/projects/mec/local/anaconda/bin/conda-server
conda-skeleton: /n/projects/mec/local/anaconda/bin/conda-skeleton
conda-token: /n/projects/mec/local/anaconda/bin/conda-token
conda-verify: /n/projects/mec/local/anaconda/bin/conda-verify
user site dirs: ~/.local/lib/python3.7
                ~/.local/lib/python2.7

CIO_TEST: <not set>
CONDA_DEFAULT_ENV: ucsc-cell-browser
CONDA_EXE: /n/projects/mec/local/anaconda/bin/conda
CONDA_PREFIX: /n/projects/mec/local/anaconda/envs/ucsc-cell-browser
CONDA_PROMPT_MODIFIER: (ucsc-cell-browser) 
CONDA_PYTHON_EXE: /n/projects/mec/local/anaconda/bin/python
CONDA_ROOT: /n/projects/mec/local/anaconda
CONDA_SHLVL: 1
CURL_CA_BUNDLE: <not set>
FASTICLIP_PATH: /n/apps/CentOS7/install/pyenv-1.0.0/pyenv/versions/2.7.5
GOPATH: /home/mec/.local/go
LD_LIBRARY_PATH: /n/apps/CentOS7/lib64:/n/apps/CentOS7/install/rbenv-0.3.0/rbenv/libexec:/n/apps/CentOS7/lib64:/n/apps/CentOS7/lib64/R/lib:/usr/lib/jvm/java-1.8.0/jre/lib/amd64/server::/n/apps/CentOS7/install/guppy-2.3.1/lib:/n/apps/CentOS7/lib:/n/apps/CentOS7/proteomics/lib
MANPATH: /home/mec/local/man:/home/mec/local/man::/n/apps/CentOS7/man:/n/apps/CentOS7/share/man:/n/apps/CentOS7/proteomics/man
MODULEPATH: /n/projects/mec/local/modulefiles:/usr/share/Modules/modulefiles:/etc/modulefiles
PATH: /n/projects/mec/local/anaconda/bin:/n/projects/mec/local/anaconda/envs/ucsc-cell-browser/bin:/home/mec/.local/bin:/n/sci/SCI-003911-GPFGRIZ/sequenceanalysis:/n/sci/SCI-003911-GPFGRIZ/sequenceanalysis/bin:/home/mec/local/bin:/n/apps/CentOS7/install/pyenv-1.0.0/pyenv/shims:/home/mec/.local/bin:/home/mec/bin:/home/mec/.opam/system/bin:/n/apps/CentOS7/install/vmatch-2.3.0/bin/vmatch-2.3.0:/n/apps/CentOS7/bin/x86_64-linux:/n/apps/CentOS7/install/rbenv-0.3.0/rbenv/shims:/n/apps/CentOS7/install/rbenv-0.3.0/rbenv/bin:/n/apps/CentOS7/install/pyenv-1.0.0/pyenv/bin:/n/apps/CentOS7/install/meme-5.4.1/libexec/meme-5.4.1:/n/apps/CentOS7/bin/blat:/n/apps/CentOS7/bin:/n/projects/mec/local/anaconda/condabin:/home/mec/.sdkman/candidates/groovy/current/bin:/home/mec/local/bin:/n/apps/CentOS7/install/nvm/versions/node/v14.17.1/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/mec/local/bin:/bin:/sbin:~mec/local/bin:~mec/local/bin
PYTHONPATH: /n/apps/CentOS7/python
QT_PLUGIN_PATH: /usr/lib64/kde4/plugins:/usr/lib/kde4/plugins
REQUESTS_CA_BUNDLE: <not set>
SSL_CERT_FILE: <not set>

FWIW: I was successful running cbBuild and browse the results using the example mini data set.

ckfromCN commented 2 years ago

I had the same problem before, I uninstalled cellbrowser in conda and then "pip install cellbrowser". Then the problem was solved

malcook commented 2 years ago

Thanks, @ckfromCN , I'm hoping to find a resolution whilst maintaining the conda install, though I may undo/redo as you suggest as a workaround.

maximilianh commented 2 years ago

We don't manage the bioconda install, but probably we should. Is there a reason Malcom why you prefer conda over pip?

On Sun, Jan 2, 2022 at 4:35 AM Malcolm Cook @.***> wrote:

Thanks, @ckfromCN https://github.com/ckfromCN , I'm hoping to find a resolution whilst maintaining the conda install, though I may undo/redo as you suggest as a workaround.

— Reply to this email directly, view it on GitHub https://github.com/maximilianh/cellBrowser/issues/233#issuecomment-1003658392, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACL4TNYFN6GANC24XRLRKLUT7BX3ANCNFSM5KVPS5OA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

malcook commented 2 years ago

I am flexing my conda muscles over pip as it appears to me to be the more generic of the two (e.g. not just addressing python).

I could I suppose use pip within a conda env.

FWIW: If I had my druthers, I would rather than conda or pip be managing scientific computing software installations with guix, but it is not sufficiently broadly used (yet?) and requires buy-in from my IT group which I don't (yet?) have. Anyway, this opinion is mainly based on reading, and not (yet?) experience.

maximilianh commented 2 years ago

I guess we could update the conda recipe through a PR. I have a note in the README that the conda recipe is usually out of date, as it's managed by the EBI and the EBI doesn't track all our releases. Let me know if this is a problem, I can look into it. Probably we should trigger a conda update whenever we release through pip (they should be a way to achieve this automatically)

On Mon, Jan 3, 2022 at 4:00 PM Malcolm Cook @.***> wrote:

I am flexing my conda muscles over pip as it appears to me to be the more generic of the two (e.g. not just addressing python).

I could I suppose use pip within a conda env.

FWIW: If I had my druthers, I would rather than conda or pip be managing scientific computing software installations with guix, but it is not sufficiently broadly used (yet?) and requires buy-in from my IT group which I don't (yet?) have. Anyway, this opinion is mainly based on reading, and not (yet?) experience.

— Reply to this email directly, view it on GitHub https://github.com/maximilianh/cellBrowser/issues/233#issuecomment-1004149574, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACL4TPBLFMXOGPLT4VZXLDUUG2YVANCNFSM5KVPS5OA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

matthewspeir commented 2 years ago

I think the conda version should be the same as pip. One of the last times I asked @pcm32 he said conda should auto-update to reflect the latest in pip.

conda installs v1.1.1 of the cell browser, which is the same as pip version (minus the pre-release versions), so it's not clear why this error pops up only for the conda version.

maximilianh commented 2 years ago

Oops, so then ignore what I said Malcolm! Sounds like Matt is much more on top of the conda questions than I am.

On Mon, Jan 3, 2022 at 5:23 PM Matt Speir @.***> wrote:

I think the conda version should be the same as pip. One of the last times I asked @pcm32 https://github.com/pcm32 he said conda should auto-update to reflect the latest in pip.

conda installs v1.1.1 of the cell browser, which is the same as pip version (minus the pre-release versions), so it's not clear why this error pops up only for the conda version.

— Reply to this email directly, view it on GitHub https://github.com/maximilianh/cellBrowser/issues/233#issuecomment-1004204776, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACL4TN7HQB7XJSRIZOPM33UUHEQJANCNFSM5KVPS5OA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

malcook commented 2 years ago

@matthewspeir if there is any improved diagnostic (or screenshare?) I can provide to help you help me further sleuth this issue, I'm at glad to oblige...

If I decide to try reinstalling (using pip or otherwise) I will report result here.

For now my successful workaround for creating custom cbs is a hybrid of R interface and CLI.

matthewspeir commented 2 years ago

I can replicate the error with the conda version. It's just not clear why the conda version would be experiencing this error when the pip version seems to be fine.

Looking at the _version.py files from the zip files for both conda and pip seem to state that they're both based on Braney's commit 7cef644bb9e843c3e0fb0d8fedce8ffa18be4212. But clearly there's something going on with how the conda file is generated that's leading to this cbImportSeurat2Cli reference leaking into the conda version as it seems the reference to to cbImportSeurat2Cli was removed all the way back in 8e8df7bba6f5b5d36438a9613da2f54440cbc733 from 2019?

@maximilianh any ideas as to why? Or do we need @pcm32 to chime in with more details about how the conda version is built and where it grabs tools like cbImportSeurat from (as they don't seem to be in the zips from neither conda or pip)?

maximilianh commented 2 years ago

The executable scripts are created by pip and conda. Weirdly, conda doesn't use the pip definitions, but has its own. Since I changed one of the function names, this broke the conda recipe. I've created a PR now on bioconda and hope that they'll add it soon which should fix this issue. Thanks Malcolm for reporting this! you can watch progress on the PR here: https://github.com/bioconda/bioconda-recipes/pull/32270

On Mon, Jan 3, 2022 at 8:43 PM Matt Speir @.***> wrote:

I can replicate the error with the conda version. It's just not clear why the conda version would be experiencing this error when the pip version seems to be fine.

Looking at the _version.py files from the zip files for both conda https://anaconda.org/bioconda/ucsc-cell-browser/1.1.1/download/noarch/ucsc-cell-browser-1.1.1-pyhdfd78af_0.tar.bz2 and pip https://files.pythonhosted.org/packages/77/54/8a19205e3cffbfb62469695619026b0a511f720c96d402a9617784753635/cellbrowser-1.1.1.tar.gz seem to state that they're both based on Braney's commit 7cef644 https://github.com/maximilianh/cellBrowser/commit/7cef644bb9e843c3e0fb0d8fedce8ffa18be4212. But clearly there's something going on with how the conda file is generated that's leading to this cbImportSeurat2Cli reference leaking into the conda version as it seems the reference to to cbImportSeurat2Cli was removed all the way back in 8e8df7b https://github.com/maximilianh/cellBrowser/commit/8e8df7bba6f5b5d36438a9613da2f54440cbc733 from 2019?

@maximilianh https://github.com/maximilianh any ideas as to why? Or do we need @pcm32 https://github.com/pcm32 to chime in with more details about how the conda version is built and where it grabs tools like cbImportSeurat from (as they don't seem to be in the zips from neither conda or pip)?

— Reply to this email directly, view it on GitHub https://github.com/maximilianh/cellBrowser/issues/233#issuecomment-1004314466, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACL4TNAE32PI4XQM3MBXXDUUH34LANCNFSM5KVPS5OA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

pcm32 commented 2 years ago

I'm sorry, I had a long holiday break. I have added some minor changes to Max's PR on bioconda, hopefully that should clear CI lint/tests and enable a merge.

pcm32 commented 2 years ago

Could someone try to see if conda installing using the artifact generated by the bioconda CI the issue gets resolved? The artifacts are here, you will need to unzip the linux download and point conda to the tar.gz within images: https://dev.azure.com/bioconda/bioconda-recipes/_build/results?buildId=9290&view=artifacts&pathAsName=false&type=publishedArtifacts (this comes from https://github.com/bioconda/bioconda-recipes/pull/32270/checks?check_run_id=4704074100 ).

maximilianh commented 2 years ago

Sorry what are artifacts? "point conda to the tar.gz within images" ? Or was this comment for someone else than me?

On Tue, Jan 4, 2022 at 5:21 PM Pablo Moreno @.***> wrote:

Could someone try to see if conda installing using the artifact generated by the bioconda CI the issue gets resolved? The artifacts are here, you will need to unzip the linux download and point conda to the tar.gz within images: https://dev.azure.com/bioconda/bioconda-recipes/_build/results?buildId=9290&view=artifacts&pathAsName=false&type=publishedArtifacts (this comes from https://github.com/bioconda/bioconda-recipes/pull/32270/checks?check_run_id=4704074100 ).

— Reply to this email directly, view it on GitHub https://github.com/maximilianh/cellBrowser/issues/233#issuecomment-1004949459, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACL4TLZG5HAEXSJBS7R2YDUUMNBJANCNFSM5KVPS5OA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

matthewspeir commented 2 years ago

I think everything seems to be fixed here? I installed the cell browser via conda and was able to run cbImportSeurat w/o arguments to see the usage message.

matthewspeir commented 2 years ago

@pcm32 We just updated Cell Browser on pypi to 1.2.0, but it doesn't look like the conda version is updated: https://anaconda.org/bioconda/ucsc-cell-browser? I was hoping to test the conda install after a new Cell Browser release just to be sure everything was fixed.

pcm32 commented 2 years ago

It might be that someone needs to approve the automatic PR with the auto bump on bioconda, did you check that?

pcm32 commented 2 years ago

I don't see the auto bump PR, when was it updated?

matthewspeir commented 2 years ago

It was updated on Friday, May 13th.

Thanks for taking a look!

matthewspeir commented 2 years ago

Well, it looks like the Cell Browser in conda still hasn't updated to 1.2.0, but it looks like the original error this issue was made for is fixed with the 1.1.1 version available, so closing this for now.