scanpy example throws error

[Jeltje]

../../cellBrowser/src/cbScanpy -e filtered_gene_bc_matrices/hg19/matrix.mtx -o myscanpyout -n pbmc3k

INFO:root:Creating myscanpyout
cbScanpy $Id: 1e7c40a1801e210aa974d42ddb5ea5f4d3f225e6 $
Input file: filtered_gene_bc_matrices/hg19/matrix.mtx
Start time: 2018-10-11 21:19:08.077474
scanpy==1.3.2 anndata==0.6.11 numpy==1.14.6 scipy==1.1.0 pandas==0.23.4 scikit-learn==0.20.0 statsmodels==0.9.0
INFO:root:Loading expression matrix: mtx format
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Data has 2700 samples/observations
Data has 32738 genes/variables
Basic filtering: keep only cells with min 200 genes
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Basic filtering: keep only gene with min 3 cells
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Remove cells with more than 0.050000 percent of mitochondrial genes
Computing percentage of mitochondrial genes
Remove cells with less than 10 and more than 15000 genes
Filtering cells
After filtering: Data has 2643 samples/observations and 13714 genes/variables
Expression normalization, counts per cell = 10000
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Variable names are not unique. To make them unique, call `.var_names_make_unique`.
Finding highly variable genes: min_mean=0.012500, max_mean=3.000000, min_disp=0.500000
Number of variable genes identified: 1844
Did log2'ing of data
Performing initial PCA, number of PCs: 100
Using 20 PCs as configured in config
Performing tSNE
Performing Louvain Clustering, using 20 PCs and 5 neighbors
  File "../../cellBrowser/src/cbPyLib/cellbrowser.py", line 3267, in cbScanpyCli
    adata = cbScanpy(matrixFname, confFname, figDir, logFname)
  File "../../cellBrowser/src/cbPyLib/cellbrowser.py", line 3190, in cbScanpy
    sc.tl.louvain(adata, resolution=res)
  File "/home/jeltjessh/venvs/cellBrowser/lib/python3.6/site-packages/scanpy/tools/louvain.py", line 103, in louvain
    g = utils.get_igraph_from_adjacency(adjacency, directed=directed)
  File "/home/jeltjessh/venvs/cellBrowser/lib/python3.6/site-packages/scanpy/utils.py", line 304, in get_igraph_from_adjacency
    import igraph as ig
> /home/jeltjessh/venvs/cellBrowser/lib/python3.6/site-packages/scanpy/utils.py(304)get_igraph_from_adjacency()
-> import igraph as ig
(Pdb)

... and then I'm stuck in the (Pdb) interface.

(Pdb) exit
INFO:root:Writing anndata object to myscanpyout/anndata.h5ad
Traceback (most recent call last):
  File "../../cellBrowser/src/cbScanpy", line 13, in <module>
    cellbrowser.cbScanpyCli()
  File "../../cellBrowser/src/cbPyLib/cellbrowser.py", line 3275, in cbScanpyCli
    adata.write(adFname)
UnboundLocalError: local variable 'adata' referenced before assignment

[maximilianh]

I’ll remove the pdb thing, that’s confusing.

So you haven’t installed igraph. Is there a reason for that? I think it’s almost required by scanpy. “pip install python-igraph” does that work for you?

On Thu, Oct 11, 2018 at 9:25 PM Jeltje notifications@github.com wrote:

../../cellBrowser/src/cbScanpy -e filtered_gene_bc_matrices/hg19/matrix.mtx -o myscanpyout -n pbmc3k

INFO:root:Creating myscanpyout cbScanpy $Id: 1e7c40a1801e210aa974d42ddb5ea5f4d3f225e6 $ Input file: filtered_gene_bc_matrices/hg19/matrix.mtx Start time: 2018-10-11 21:19:08.077474 scanpy==1.3.2 anndata==0.6.11 numpy==1.14.6 scipy==1.1.0 pandas==0.23.4 scikit-learn==0.20.0 statsmodels==0.9.0 INFO:root:Loading expression matrix: mtx format Variable names are not unique. To make them unique, call .var_names_make_unique. Data has 2700 samples/observations Data has 32738 genes/variables Basic filtering: keep only cells with min 200 genes Variable names are not unique. To make them unique, call .var_names_make_unique. Variable names are not unique. To make them unique, call .var_names_make_unique. Basic filtering: keep only gene with min 3 cells Variable names are not unique. To make them unique, call .var_names_make_unique. Variable names are not unique. To make them unique, call .var_names_make_unique. Remove cells with more than 0.050000 percent of mitochondrial genes Computing percentage of mitochondrial genes Remove cells with less than 10 and more than 15000 genes Filtering cells After filtering: Data has 2643 samples/observations and 13714 genes/variables Expression normalization, counts per cell = 10000 Variable names are not unique. To make them unique, call .var_names_make_unique. Variable names are not unique. To make them unique, call .var_names_make_unique. Variable names are not unique. To make them unique, call .var_names_make_unique. Variable names are not unique. To make them unique, call .var_names_make_unique. Finding highly variable genes: min_mean=0.012500, max_mean=3.000000, min_disp=0.500000 Number of variable genes identified: 1844 Did log2'ing of data Performing initial PCA, number of PCs: 100 Using 20 PCs as configured in config Performing tSNE Performing Louvain Clustering, using 20 PCs and 5 neighbors File "../../cellBrowser/src/cbPyLib/cellbrowser.py", line 3267, in cbScanpyCli adata = cbScanpy(matrixFname, confFname, figDir, logFname) File "../../cellBrowser/src/cbPyLib/cellbrowser.py", line 3190, in cbScanpy sc.tl.louvain(adata, resolution=res) File "/home/jeltjessh/venvs/cellBrowser/lib/python3.6/site-packages/scanpy/tools/louvain.py", line 103, in louvain g = utils.get_igraph_from_adjacency(adjacency, directed=directed) File "/home/jeltjessh/venvs/cellBrowser/lib/python3.6/site-packages/scanpy/utils.py", line 304, in get_igraph_from_adjacency import igraph as ig

/home/jeltjessh/venvs/cellBrowser/lib/python3.6/site-packages/scanpy/utils.py(304)get_igraph_from_adjacency() -> import igraph as ig (Pdb)

... and then I'm stuck in the (Pdb) interface.

(Pdb) exit INFO:root:Writing anndata object to myscanpyout/anndata.h5ad Traceback (most recent call last): File "../../cellBrowser/src/cbScanpy", line 13, in cellbrowser.cbScanpyCli() File "../../cellBrowser/src/cbPyLib/cellbrowser.py", line 3275, in cbScanpyCli adata.write(adFname) UnboundLocalError: local variable 'adata' referenced before assignment

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/maximilianh/cellBrowser/issues/42, or mute the thread https://github.com/notifications/unsubscribe-auth/AAS-TV3vlj0hvLsXoCLJ3LQpsQDYSFuuks5ukBmmgaJpZM4XYxKw .

[Jeltje]

igraph isn't a dependency of scanpy:

pip show scanpy
Name: scanpy
Version: 1.3.2
Summary: Single-Cell Analysis in Python.
Home-page: http://github.com/theislab/scanpy
Author: Alex Wolf, Philipp Angerer, Davide Cittaro, Gokcen Eraslan, Fidel Ramirez, Tobias Callies
Author-email: alex.wolf@helmholtz-muenchen.de
License: BSD
Location: /home/jeltjessh/venvs/cellBrowser/lib/python3.6/site-packages
Requires: numba, natsort, matplotlib, joblib, pandas, h5py, scipy, seaborn, networkx, tables, statsmodels, scikit-learn, anndata

So I pip installed it, but then I got

../../cellBrowser/src/cbScanpy -e filtered_gene_bc_matrices/hg19/matrix.mtx -o myscanpyout -n pbmc3k 2>&1 | tee cb.log
INFO:root:Creating myscanpyout
  File "../../cellBrowser/src/cbPyLib/cellbrowser.py", line 3267, in cbScanpyCli
    adata = cbScanpy(matrixFname, confFname, figDir, logFname)
  File "../../cellBrowser/src/cbPyLib/cellbrowser.py", line 3064, in cbScanpy
    sc.logging.print_versions()
  File "/home/jeltjessh/venvs/cellBrowser/lib/python3.6/site-packages/scanpy/logging.py", line 180, in print_versions
    _print_versions_dependencies(['scanpy'] + _DEPENDENCIES_NUMERICS)
  File "/home/jeltjessh/venvs/cellBrowser/lib/python3.6/site-packages/scanpy/logging.py", line 169, in _print_versions_dependencies
    imp = __import__(mod_name)
  File "/home/jeltjessh/venvs/cellBrowser/lib/python3.6/site-packages/igraph/__init__.py", line 8, in <module>
    raise DeprecationWarning("To avoid name collision with the igraph project, "
cbScanpy $Id: 1e7c40a1801e210aa974d42ddb5ea5f4d3f225e6 $
Input file: filtered_gene_bc_matrices/hg19/matrix.mtx
Start time: 2018-10-12 10:22:04.242416
scanpy==1.3.2 anndata==0.6.11 numpy==1.14.6 scipy==1.1.0 pandas==0.23.4 scikit-learn==0.20.0 statsmodels==0.9.0 > /home/jeltjessh/venvs/cellBrowser/lib/python3.6/site-packages/igraph/__init__.py(8)<module>()
-> raise DeprecationWarning("To avoid name collision with the igraph project, "

It's a cliffhanger! It's listing the installed versions but not telling me what it wants.

Anyway, it then occurred to me you might have changed things since yesterday, so I pulled the latest.

../../cellBrowser/src/cbScanpy -e filtered_gene_bc_matrices/hg19/matrix.mtx -o myscanpyout -n pbmc3k 
Traceback (most recent call last):
  File "../../cellBrowser/src/cbScanpy", line 13, in <module>
    cellbrowser.cbScanpyCli()
  File "../../cellBrowser/src/cbPyLib/cellbrowser/cellbrowser.py", line 2950, in cbScanpyCli
    if options.init:
AttributeError: 'Values' object has no attribute 'init'

Can you let me know when you have tested the example on a clean system?

[maximilianh]

I accidentally did a commit yesterday where I should have not committed. very sorry. I'm trying to fix things now or will revert.

[maximilianh]

I think I've fixed it all. I'll do test it on someone else's laptop today, but otherwise the following should work:

pip install cellbrowser

If you're not root and just a normal Unix user, do this:

pip install cellbrowser --user

On Sat, Oct 13, 2018 at 12:45 PM Maximilian Haeussler maximilianh@gmail.com wrote:

I accidentally did a commit yesterday where I should have not committed. very sorry. I'm trying to fix things now or will revert.

[maximilianh]

OK, now getting back to your igraph question:

did you really install the correct package:

pip install igraph-python

On Sat, Oct 13, 2018 at 2:41 PM Maximilian Haeussler maximilianh@gmail.com wrote:

I think I've fixed it all. I'll do test it on someone else's laptop today, but otherwise the following should work:

pip install cellbrowser

If you're not root and just a normal Unix user, do this:

pip install cellbrowser --user

On Sat, Oct 13, 2018 at 12:45 PM Maximilian Haeussler maximilianh@gmail.com wrote:

I accidentally did a commit yesterday where I should have not committed. very sorry. I'm trying to fix things now or will revert.

[maximilianh]

I don't know why igraph is not a requirement for scanpy. De facto, it is required to do anything useful with scanpy. I think Alex doesn't want it as a requirement as very often the scanpy installation will fail, as the igraph package itself has so many dependencies.

His recommended pip command is this one and it does have the requirements:

pip install scanpy python-igraph louvain

Are you using conda? If there is any problem with this pip command, try using conda...

Let me know how it goes, I've been through this, too. It's the biggest stumbling block with scanpy.

If everything goes according to plan, you'll soon have a seurat wrapper, too. On Sat, Oct 13, 2018 at 2:42 PM Maximilian Haeussler maximilianh@gmail.com wrote:

OK, now getting back to your igraph question:

did you really install the correct package:

pip install igraph-python

On Sat, Oct 13, 2018 at 2:41 PM Maximilian Haeussler maximilianh@gmail.com wrote:

I think I've fixed it all. I'll do test it on someone else's laptop today, but otherwise the following should work:

pip install cellbrowser

If you're not root and just a normal Unix user, do this:

pip install cellbrowser --user

On Sat, Oct 13, 2018 at 12:45 PM Maximilian Haeussler maximilianh@gmail.com wrote:

I accidentally did a commit yesterday where I should have not committed. very sorry. I'm trying to fix things now or will revert.

[maximilianh]

Closing this one, too, as #11 was closed