search

maximilianh / cellBrowser

main repo: https://github.com/ucscGenomeBrowser/cellBrowser/ - Python pipeline and Javascript scatter plot library for single-cell datasets, http://cellbrowser.rtfd.org
https://github.com/ucscGenomeBrowser/cellBrowser/
GNU General Public License v3.0
104 stars 40 forks source link

cbScanpy dies on PAGA #56

Closed Jeltje closed 5 years ago

Jeltje commented 5 years ago

This is version 0.4.23 cbScanpy -e </path/to/my.mtx> -o ipf3_files -n ipf3 gets 

Performing UMAP
Performing PHATE
Performing PAGA+ForceAtlas2
Traceback (most recent call last):
  File "/home/cellranger/anaconda3/envs/cbConda/bin/cbScanpy", line 11, in <module>
    sys.exit(cbScanpyCli())
  File "/home/cellranger/anaconda3/envs/cbConda/lib/python3.6/site-packages/cellbrowser/cellbrowser.py", line 3425, in cbScanpyCli
    adata = cbScanpy(matrixFname, confFname, figDir, logFname)
  File "/home/cellranger/anaconda3/envs/cbConda/lib/python3.6/site-packages/cellbrowser/cellbrowser.py", line 3331, in cbScanpy
    adata.obsm["X_pagaFa"] = adata.obsm["X_draw_graph_fa"]
  File "/home/cellranger/anaconda3/envs/cbConda/lib/python3.6/site-packages/numpy/core/records.py", line 500, in __getitem__
    obj = super(recarray, self).__getitem__(indx)
ValueError: no field of name X_draw_graph_fa
Jeltje commented 5 years ago

I then tried a run with this: doLayouts = ['umap', 'phate', 'fa', 'fr', 'grid_fr', 'kk', 'lgl', 'drl', 'rt'] and that died with

ERROR:root:layout name grid_fr is not valid. Valid layouts: {'fa': 'ForceAtlas2', 'fr': 'Fruchterman Reingold', 'kk': 'Kamadi Kawai', 'drl': 'DrL Distributed Recursive Layout', 'tsne': 't-SNE', 'umap': 'UMAP', 'pagaFa': 'PAGA/ForceAtlas2', 'pagaFr': 'PAGA/Fruchterman-Reingold', 'phate': 'PHATE'}

Note that lgl and rt are also not listed as a valid layout, though they are listed as 'possible options' in scanpy.conf

maximilianh commented 5 years ago

I'm leaving this open for now until I hear back from you.

maximilianh commented 5 years ago

BTW I've successfully run cbScanpy using doLayouts="all" right now on pbmc_small

maximilianh commented 5 years ago

I know the first problem: ValueError: no field of name X_draw_graph_fa I thought I had made sure that this happens anymore. I wonder if it has to do with the scanpy version or something like that. I've added one more option in a commit a few days ago, so hopefully this will not be a problem anymore. Let me know if it still is. (scanpy changed the name of the

second problem: sorry you got bitten by my most recent change, I didn't update the docs. I changed it, so this should work now. release 0.4.24

maximilianh commented 5 years ago

cbScanpy apparently worked for you now.

Jeltje commented 5 years ago

Looks like I can't reopen this issue, but I just pip installed 0.4.24 and still get

Performing PAGA+ForceAtlas2
Traceback (most recent call last):
  File "/home/cellranger/anaconda3/envs/cellBrowserConda/bin/cbScanpy", line 11, in <module>
    sys.exit(cbScanpyCli())
  File "/home/cellranger/anaconda3/envs/cellBrowserConda/lib/python3.6/site-packages/cellbrowser/cellbrowser.py", line 3435, in cbScanpyCli
    adata = cbScanpy(matrixFname, confFname, figDir, logFname)
  File "/home/cellranger/anaconda3/envs/cellBrowserConda/lib/python3.6/site-packages/cellbrowser/cellbrowser.py", line 3341, in cbScanpy
    adata.obsm["X_pagaFa"] = adata.obsm["X_draw_graph_fa"]
  File "/home/cellranger/anaconda3/envs/cellBrowserConda/lib/python3.6/site-packages/numpy/core/records.py", line 500, in __getitem__
    obj = super(recarray, self).__getitem__(indx)
ValueError: no field of name X_draw_graph_fa

This happens with scanpy 1.3.2 and also after pip --upgrade scanpy with 1.3.6

maximilianh commented 5 years ago

Argh, this is frustrating. I'm surprised that I can't reproduce this problem. So you're using cbScanpy? How did you set the doLayouts option? to "all" ? On Thu, Dec 13, 2018 at 1:12 AM Jeltje notifications@github.com wrote:

Looks like I can't reopen this issue, but I just pip installed 0.4.24 and still get

Performing PAGA+ForceAtlas2 Traceback (most recent call last): File "/home/cellranger/anaconda3/envs/cellBrowserConda/bin/cbScanpy", line 11, in sys.exit(cbScanpyCli()) File "/home/cellranger/anaconda3/envs/cellBrowserConda/lib/python3.6/site-packages/cellbrowser/cellbrowser.py", line 3435, in cbScanpyCli adata = cbScanpy(matrixFname, confFname, figDir, logFname) File "/home/cellranger/anaconda3/envs/cellBrowserConda/lib/python3.6/site-packages/cellbrowser/cellbrowser.py", line 3341, in cbScanpy adata.obsm["X_pagaFa"] = adata.obsm["X_draw_graph_fa"] File "/home/cellranger/anaconda3/envs/cellBrowserConda/lib/python3.6/site-packages/numpy/core/records.py", line 500, in getitem obj = super(recarray, self).getitem(indx) ValueError: no field of name X_draw_graph_fa

This happens with scanpy 1.3.2 and also after pip --upgrade scanpy with 1.3.6

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

Jeltje commented 5 years ago

Yes cbScanpy and yes "all". To get it to work I replaced the recommendedLayouts line in cellbrowser.py with one that excludes PAGA and then it runs. This is the full list of dependencies as printed at the top of the log: scanpy==1.3.6 anndata==0.6.15 numpy==1.14.0 scipy==1.1.0 pandas==0.23.4 scikit-learn==0.20.0 statsmodels==0.9.0 python-igraph==0.7.1 louvain==0.6.1

I did pip --upgrade on all of these and ran with doLayouts=["pagaFa"]. Same error. scanpy==1.3.6 anndata==0.6.15 numpy==1.15.4 scipy==1.1.0 pandas==0.23.4 scikit-learn==0.20.1 statsmodels==0.9.0 python-igraph==0.7.1 louvain==0.6.1

Note that the actual error comes from numpy/core/records.py so I thought the upgrade would help...

maximilianh commented 5 years ago

I still can't reproduce this, but I'm checking now if X_draw_graph_fa really exists instead of just assuming that it exists.

The code outputs a warning if it doesn't exist and it will simply skip this layout, instead of failing, if the name produced by scanpy does not match what is in the current documentation.

Thanks Jeltje!

On Thu, Dec 13, 2018 at 8:01 PM Jeltje notifications@github.com wrote:

Yes cbScanpy and yes "all". To get it to work I replaced the recommendedLayouts line in cellbrowser.py with one that excludes PAGA and then it runs. This is the full list of dependencies as printed at the top of the log: scanpy==1.3.6 anndata==0.6.15 numpy==1.14.0 scipy==1.1.0 pandas==0.23.4 scikit-learn==0.20.0 statsmodels==0.9.0 python-igraph==0.7.1 louvain==0.6.1

I did pip --upgrade on all of these and ran with doLayouts=["pagaFa"]. Same error. scanpy==1.3.6 anndata==0.6.15 numpy==1.15.4 scipy==1.1.0 pandas==0.23.4 scikit-learn==0.20.1 statsmodels==0.9.0 python-igraph==0.7.1 louvain==0.6.1

Note that the actual error comes from numpy/core/records.py so I thought the upgrade would help...

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/maximilianh/cellBrowser/issues/56#issuecomment-447081690, or mute the thread https://github.com/notifications/unsubscribe-auth/AAS-TX4MHvAnEsuZZ6z5DfZDoZVs7NURks5u4qPygaJpZM4ZB3x1 .