search

aertslab / SCENIC

SCENIC is an R package to infer Gene Regulatory Networks and cell types from single-cell RNA-seq data.
http://scenic.aertslab.org
GNU General Public License v3.0
422 stars 94 forks source link

Handling GRNboost output #40

Open dpcook opened 6 years ago

dpcook commented 6 years ago

Hi there. I've been going through the vignettes and I can't seem to figure out how to get scenicOptions to point to the tsv output file from GRNboost (or a data frame of the results after importing it into R).

Any advice on how to use GRNboost results in the SCENIC pipeline would be greatly appreciated!

JBreunig commented 6 years ago

I think that we are waiting for this function to be implemented: https://github.com/aertslab/SCENIC/blob/master/man/importGRNBoostResults.Rd

Also referred to here: https://htmlpreview.github.io/?https://github.com/aertslab/SCENIC/blob/master/inst/doc/SCENIC_Running.html#option-b-grnboost-python

We are now ready to run GRNBoost. You can find the tutorial with some examples in https://arboreto.readthedocs.io/en/latest/examples.html. Once it has finished running, you can import the output from GRNBoost back to R with:

importGRNBoostResults() # TODO (Not implemented yet!)

I've found SCENIC tremendously useful but I look forward to being able to skipping the GENIE3 step as it is very time-consuming compared with what we've seen with GRNBoost.

jpezoldt commented 6 years ago

What worked for me was to rename the colnames of the GRNBoost output .tsv required by the downstream wrapper runSCENIC_1_coexNetwork2modules() and save the file in the respective scenic int/ directory.

e.g.: GRNBoost_output <- read.delim("~dir_to_GRNboost_tsv_file/ex_5000SC_network.tsv", header=FALSE) colnames(GRNBoost_output) <- c("TF","Target","weight") saveRDS(GRNBoost_output, file="int/1.4_GENIE3_linkList.Rds")

JBreunig commented 6 years ago

Thanks for the suggestion! I tried that and ended up with the following error:

runSCENIC_1_coexNetwork2modules(scenicOptions) 17:14 Creating TF modules Error in quantile.default(linkList$weight, probs = c(0.75, 0.9)) : factors are not allowed

It seems like the GENIE3 weights are <1 while the pySCENIC weights/importance range from .02-160

jpezoldt commented 6 years ago

Indeed GRNboost has a different range for the scores as compared to GENIE3. But so far I have not observed an impact for my analysis and it also should not have one.

It seems like the GENIE3 weights are <1 while the pySCENIC weights/importance range from .02-160

I checked, whether having the class factor in the GRNBoost_output table will give me the same error message. Indeed, this is the case and if the "weight" column is class() factor I get the identical error message. You could check your input table #columns for the class() and convert the weight column to numeric. Hope this helps.

KabitaBaral1 commented 4 years ago

Hi @dpcook @JBreunig @jpezoldt , My dataset is too big for GENIE3 and I have been trying to run GRNboost using the vignette as suggested in https://arboreto.readthedocs.io/en/latest/examples.html

However, I feel like I have missed something because my code does not seem to run. Could you please help me with which output from SCENIC do you input into GRNboost ?

1.1_genesKept.Rds 1.2_corrMat.Rds cellInfo.Rds colVars.Rds or scenicOptions.Rds?

Thank you

JBreunig commented 4 years ago

I moved on to a scanpy-->pyscenic workflow. I don't believe that I ever got things to work going from pySCENIC to SCENIC but that was before the tutorials were written.

KabitaBaral1 commented 4 years ago

Hi @JBreunig Thank you.

I was wondering if you used Jupyter Notebook or CLI for grn step of pyscenic? I am using Jupyter Notebook and I keep running into weird errors in this step:

!pyscenic grn {f_loom_path_scenic} {f_tfs} -o adj.csv --num_workers 20

I am not sure if it is version compatibility issues with dependencies and dask or something else. I followed the following protocol: https://github.com/aertslab/SCENICprotocol/blob/master/notebooks/PBMC10k_SCENIC-protocol-CLI.ipynb

Thank you

JBreunig commented 4 years ago

I use spyder. But for that particular step, I just go back to the CLI and run it as a typical linux command (from the appropriate directory).

pyscenic grn matrix.csv mm_mgi_tfs.txt -o matrix.adjacencies.tsv --num_workers 15

KabitaBaral1 commented 4 years ago

Thank you @JBreunig Just one more question, which version of dask, distributed, pandas, msgpack and tornado do you use? I seem to get errors associated with version incompatibility even when I run it via commandline.

especially this for tornado: tornado.iostream.StreamClosedError: Stream is closed

Thanks

JBreunig commented 4 years ago

It looks like: dask - 1.0.0, distributed - 1.28.1, pandas - 0.25.3, msgpack - 0.6.1 and tornado 6.0.3

KabitaBaral1 commented 4 years ago

Thank you very much @JBreunig.

Hopefully it works. Thanks again

KabitaBaral1 commented 4 years ago

Hi @JBreunig Sorry to bug again, I tried running my code in linux and I get this very long error. Did you ever run into something like this? Could you please help me figure this out?

Thank you

/home/kbaral/anaconda3/lib/python3.7/site-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. data = yaml.load(f.read()) or {}

2020-04-10 19:58:06,483 - pyscenic.cli.pyscenic - INFO - Loading expression matrix.

2020-04-10 19:58:12,336 - pyscenic.cli.pyscenic - INFO - Inferring regulatory networks. /home/kbaral/anaconda3/lib/python3.7/site-packages/dask/config.py:161: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. data = yaml.load(f.read()) or {} preparing dask client parsing input /home/kbaral/anaconda3/lib/python3.7/site-packages/arboreto/algo.py:214: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead. expression_matrix = expression_data.as_matrix() creating dask graph 20 partitions computing dask graph 'infer_data failed for target KIAA2013' Retry (1/10). Failure caused by ValueError("Regression for target gene KIAA2013 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC103591.3' Retry (1/10). Failure caused by ValueError("Regression for target gene AC103591.3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target FBXO6' Retry (1/10). Failure caused by ValueError("Regression for target gene FBXO6 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ODF2L' Retry (1/10). Failure caused by ValueError("Regression for target gene ODF2L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC103591.3' Retry (2/10). Failure caused by ValueError("Regression for target gene AC103591.3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target KIAA2013' Retry (2/10). Failure caused by ValueError("Regression for target gene KIAA2013 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target FBXO6' Retry (2/10). Failure caused by ValueError("Regression for target gene FBXO6 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ODF2L' Retry (2/10). Failure caused by ValueError("Regression for target gene ODF2L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092343.1' Retry (1/10). Failure caused by ValueError("Regression for target gene AC092343.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092807.2' Retry (1/10). Failure caused by ValueError("Regression for target gene AC092807.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target BMP8A' Retry (1/10). Failure caused by ValueError("Regression for target gene BMP8A failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AGTRAP' Retry (1/10). Failure caused by ValueError("Regression for target gene AGTRAP failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC103591.3' Retry (3/10). Failure caused by ValueError("Regression for target gene AC103591.3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target KIAA2013' Retry (3/10). Failure caused by ValueError("Regression for target gene KIAA2013 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target GIPC2' Retry (1/10). Failure caused by ValueError("Regression for target gene GIPC2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target FBXO6' Retry (3/10). Failure caused by ValueError("Regression for target gene FBXO6 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PIK3CD' Retry (1/10). Failure caused by ValueError("Regression for target gene PIK3CD failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ODF2L' Retry (3/10). Failure caused by ValueError("Regression for target gene ODF2L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AK5' Retry (1/10). Failure caused by ValueError("Regression for target gene AK5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092343.1' Retry (2/10). Failure caused by ValueError("Regression for target gene AC092343.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092807.2' Retry (2/10). Failure caused by ValueError("Regression for target gene AC092807.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CSMD2' Retry (1/10). Failure caused by ValueError("Regression for target gene CSMD2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target BMP8A' Retry (2/10). Failure caused by ValueError("Regression for target gene BMP8A failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AGTRAP' Retry (2/10). Failure caused by ValueError("Regression for target gene AGTRAP failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC103591.3' Retry (4/10). Failure caused by ValueError("Regression for target gene AC103591.3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target KIAA2013' Retry (4/10). Failure caused by ValueError("Regression for target gene KIAA2013 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL031429.1' Retry (1/10). Failure caused by ValueError("Regression for target gene AL031429.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target LRRC38' Retry (1/10). Failure caused by ValueError("Regression for target gene LRRC38 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target SLC44A5' Retry (1/10). Failure caused by ValueError("Regression for target gene SLC44A5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092807.2' Retry (3/10). Failure caused by ValueError("Regression for target gene AC092807.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ODF2L' Retry (4/10). Failure caused by ValueError("Regression for target gene ODF2L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CSMD2' Retry (2/10). Failure caused by ValueError("Regression for target gene CSMD2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092343.1' Retry (3/10). Failure caused by ValueError("Regression for target gene AC092343.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target GIPC2' Retry (2/10). Failure caused by ValueError("Regression for target gene GIPC2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target BMP8A' Retry (3/10). Failure caused by ValueError("Regression for target gene BMP8A failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AK5' Retry (2/10). Failure caused by ValueError("Regression for target gene AK5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AGTRAP' Retry (3/10). Failure caused by ValueError("Regression for target gene AGTRAP failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target FBXO6' Retry (4/10). Failure caused by ValueError("Regression for target gene FBXO6 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC103591.3' Retry (5/10). Failure caused by ValueError("Regression for target gene AC103591.3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PIK3CD' Retry (2/10). Failure caused by ValueError("Regression for target gene PIK3CD failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target KIAA2013' Retry (5/10). Failure caused by ValueError("Regression for target gene KIAA2013 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target C1orf109' Retry (1/10). Failure caused by ValueError("Regression for target gene C1orf109 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target LRRC38' Retry (2/10). Failure caused by ValueError("Regression for target gene LRRC38 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL031429.1' Retry (2/10). Failure caused by ValueError("Regression for target gene AL031429.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC093158.1' Retry (1/10). Failure caused by ValueError("Regression for target gene AC093158.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TMCO4' Retry (1/10). Failure caused by ValueError("Regression for target gene TMCO4 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092807.2' Retry (4/10). Failure caused by ValueError("Regression for target gene AC092807.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092343.1' Retry (4/10). Failure caused by ValueError("Regression for target gene AC092343.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CSMD2' Retry (3/10). Failure caused by ValueError("Regression for target gene CSMD2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AK5' Retry (3/10). Failure caused by ValueError("Regression for target gene AK5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ODF2L' Retry (5/10). Failure caused by ValueError("Regression for target gene ODF2L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target SLC44A5' Retry (2/10). Failure caused by ValueError("Regression for target gene SLC44A5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target BMP8A' Retry (4/10). Failure caused by ValueError("Regression for target gene BMP8A failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC103591.3' Retry (6/10). Failure caused by ValueError("Regression for target gene AC103591.3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target GIPC2' Retry (3/10). Failure caused by ValueError("Regression for target gene GIPC2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target C1orf109' Retry (2/10). Failure caused by ValueError("Regression for target gene C1orf109 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL031429.1' Retry (3/10). Failure caused by ValueError("Regression for target gene AL031429.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target FBXO6' Retry (5/10). Failure caused by ValueError("Regression for target gene FBXO6 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target LRRC38' Retry (3/10). Failure caused by ValueError("Regression for target gene LRRC38 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target KIAA2013' Retry (6/10). Failure caused by ValueError("Regression for target gene KIAA2013 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AGTRAP' Retry (4/10). Failure caused by ValueError("Regression for target gene AGTRAP failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC093158.1' Retry (2/10). Failure caused by ValueError("Regression for target gene AC093158.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PIK3CD' Retry (3/10). Failure caused by ValueError("Regression for target gene PIK3CD failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TMCO4' Retry (2/10). Failure caused by ValueError("Regression for target gene TMCO4 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092343.1' Retry (5/10). Failure caused by ValueError("Regression for target gene AC092343.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092807.2' Retry (5/10). Failure caused by ValueError("Regression for target gene AC092807.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CSMD2' Retry (4/10). Failure caused by ValueError("Regression for target gene CSMD2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AK5' Retry (4/10). Failure caused by ValueError("Regression for target gene AK5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target SLC44A5' Retry (3/10). Failure caused by ValueError("Regression for target gene SLC44A5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC103591.3' Retry (7/10). Failure caused by ValueError("Regression for target gene AC103591.3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target LRRC38' Retry (4/10). Failure caused by ValueError("Regression for target gene LRRC38 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target C1orf109' Retry (3/10). Failure caused by ValueError("Regression for target gene C1orf109 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ODF2L' Retry (6/10). Failure caused by ValueError("Regression for target gene ODF2L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target GIPC2' Retry (4/10). Failure caused by ValueError("Regression for target gene GIPC2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC093158.1' Retry (3/10). Failure caused by ValueError("Regression for target gene AC093158.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AGTRAP' Retry (5/10). Failure caused by ValueError("Regression for target gene AGTRAP failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target BMP8A' Retry (5/10). Failure caused by ValueError("Regression for target gene BMP8A failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL031429.1' Retry (4/10). Failure caused by ValueError("Regression for target gene AL031429.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target FBXO6' Retry (6/10). Failure caused by ValueError("Regression for target gene FBXO6 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PIK3CD' Retry (4/10). Failure caused by ValueError("Regression for target gene PIK3CD failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092343.1' Retry (6/10). Failure caused by ValueError("Regression for target gene AC092343.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target KIAA2013' Retry (7/10). Failure caused by ValueError("Regression for target gene KIAA2013 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TMCO4' Retry (3/10). Failure caused by ValueError("Regression for target gene TMCO4 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CSMD2' Retry (5/10). Failure caused by ValueError("Regression for target gene CSMD2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092807.2' Retry (6/10). Failure caused by ValueError("Regression for target gene AC092807.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AK5' Retry (5/10). Failure caused by ValueError("Regression for target gene AK5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target LRRC38' Retry (5/10). Failure caused by ValueError("Regression for target gene LRRC38 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target SLC44A5' Retry (4/10). Failure caused by ValueError("Regression for target gene SLC44A5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target C1orf109' Retry (4/10). Failure caused by ValueError("Regression for target gene C1orf109 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target GIPC2' Retry (5/10). Failure caused by ValueError("Regression for target gene GIPC2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC093158.1' Retry (4/10). Failure caused by ValueError("Regression for target gene AC093158.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AGTRAP' Retry (6/10). Failure caused by ValueError("Regression for target gene AGTRAP failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ODF2L' Retry (7/10). Failure caused by ValueError("Regression for target gene ODF2L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PIK3CD' Retry (5/10). Failure caused by ValueError("Regression for target gene PIK3CD failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092343.1' Retry (7/10). Failure caused by ValueError("Regression for target gene AC092343.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC103591.3' Retry (8/10). Failure caused by ValueError("Regression for target gene AC103591.3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CSMD2' Retry (6/10). Failure caused by ValueError("Regression for target gene CSMD2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092807.2' Retry (7/10). Failure caused by ValueError("Regression for target gene AC092807.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TMCO4' Retry (4/10). Failure caused by ValueError("Regression for target gene TMCO4 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AK5' Retry (6/10). Failure caused by ValueError("Regression for target gene AK5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target SLC44A5' Retry (5/10). Failure caused by ValueError("Regression for target gene SLC44A5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target BMP8A' Retry (6/10). Failure caused by ValueError("Regression for target gene BMP8A failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL031429.1' Retry (5/10). Failure caused by ValueError("Regression for target gene AL031429.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target C1orf109' Retry (5/10). Failure caused by ValueError("Regression for target gene C1orf109 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target FBXO6' Retry (7/10). Failure caused by ValueError("Regression for target gene FBXO6 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target GIPC2' Retry (6/10). Failure caused by ValueError("Regression for target gene GIPC2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC093158.1' Retry (5/10). Failure caused by ValueError("Regression for target gene AC093158.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target KIAA2013' Retry (8/10). Failure caused by ValueError("Regression for target gene KIAA2013 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092343.1' Retry (8/10). Failure caused by ValueError("Regression for target gene AC092343.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target LRRC38' Retry (6/10). Failure caused by ValueError("Regression for target gene LRRC38 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AGTRAP' Retry (7/10). Failure caused by ValueError("Regression for target gene AGTRAP failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PIK3CD' Retry (6/10). Failure caused by ValueError("Regression for target gene PIK3CD failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ODF2L' Retry (8/10). Failure caused by ValueError("Regression for target gene ODF2L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CSMD2' Retry (7/10). Failure caused by ValueError("Regression for target gene CSMD2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092807.2' Retry (8/10). Failure caused by ValueError("Regression for target gene AC092807.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AK5' Retry (7/10). Failure caused by ValueError("Regression for target gene AK5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TMCO4' Retry (5/10). Failure caused by ValueError("Regression for target gene TMCO4 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC103591.3' Retry (9/10). Failure caused by ValueError("Regression for target gene AC103591.3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target SLC44A5' Retry (6/10). Failure caused by ValueError("Regression for target gene SLC44A5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target GIPC2' Retry (7/10). Failure caused by ValueError("Regression for target gene GIPC2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC093158.1' Retry (6/10). Failure caused by ValueError("Regression for target gene AC093158.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092343.1' Retry (9/10). Failure caused by ValueError("Regression for target gene AC092343.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target C1orf109' Retry (6/10). Failure caused by ValueError("Regression for target gene C1orf109 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AGTRAP' Retry (8/10). Failure caused by ValueError("Regression for target gene AGTRAP failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PIK3CD' Retry (7/10). Failure caused by ValueError("Regression for target gene PIK3CD failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target LRRC38' Retry (7/10). Failure caused by ValueError("Regression for target gene LRRC38 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ODF2L' Retry (9/10). Failure caused by ValueError("Regression for target gene ODF2L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092807.2' Retry (9/10). Failure caused by ValueError("Regression for target gene AC092807.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target BMP8A' Retry (7/10). Failure caused by ValueError("Regression for target gene BMP8A failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL031429.1' Retry (6/10). Failure caused by ValueError("Regression for target gene AL031429.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target FBXO6' Retry (8/10). Failure caused by ValueError("Regression for target gene FBXO6 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TMCO4' Retry (6/10). Failure caused by ValueError("Regression for target gene TMCO4 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CSMD2' Retry (8/10). Failure caused by ValueError("Regression for target gene CSMD2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC103591.3' Retry (10/10). Failure caused by ValueError("Regression for target gene AC103591.3 failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fa158>, (<function infer_partial_network at 0x7f22387e2268>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

'infer_data failed for target KIAA2013' Retry (9/10). Failure caused by ValueError("Regression for target gene KIAA2013 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target GIPC2' Retry (8/10). Failure caused by ValueError("Regression for target gene GIPC2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target SLC44A5' Retry (7/10). Failure caused by ValueError("Regression for target gene SLC44A5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC093158.1' Retry (7/10). Failure caused by ValueError("Regression for target gene AC093158.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC092343.1' Retry (10/10). Failure caused by ValueError("Regression for target gene AC092343.1 failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fa158>, (<function infer_partial_network at 0x7f22387e2268>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

'infer_data failed for target AGTRAP' Retry (9/10). Failure caused by ValueError("Regression for target gene AGTRAP failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target LRRC38' Retry (8/10). Failure caused by ValueError("Regression for target gene LRRC38 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ODF2L' Retry (10/10). Failure caused by ValueError("Regression for target gene ODF2L failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fa158>, (<function infer_partial_network at 0x7f22387e2268>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

'infer_data failed for target AC092807.2' Retry (10/10). Failure caused by ValueError("Regression for target gene AC092807.2 failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fa158>, (<function infer_partial_network at 0x7f22387e2268>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

'infer_data failed for target PIK3CD' Retry (8/10). Failure caused by ValueError("Regression for target gene PIK3CD failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AK5' Retry (8/10). Failure caused by ValueError("Regression for target gene AK5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target BMP8A' Retry (8/10). Failure caused by ValueError("Regression for target gene BMP8A failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TMCO4' Retry (7/10). Failure caused by ValueError("Regression for target gene TMCO4 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CSMD2' Retry (9/10). Failure caused by ValueError("Regression for target gene CSMD2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target GIPC2' Retry (9/10). Failure caused by ValueError("Regression for target gene GIPC2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target C1orf109' Retry (7/10). Failure caused by ValueError("Regression for target gene C1orf109 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CNTN2' Retry (1/10). Failure caused by ValueError("Regression for target gene CNTN2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC093158.1' Retry (8/10). Failure caused by ValueError("Regression for target gene AC093158.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL031429.1' Retry (7/10). Failure caused by ValueError("Regression for target gene AL031429.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL391832.2' Retry (1/10). Failure caused by ValueError("Regression for target gene AL391832.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AGTRAP' Retry (10/10). Failure caused by ValueError("Regression for target gene AGTRAP failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fa158>, (<function infer_partial_network at 0x7f22387e2268>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

'infer_data failed for target FBXO6' Retry (9/10). Failure caused by ValueError("Regression for target gene FBXO6 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target B4GALT3' Retry (1/10). Failure caused by ValueError("Regression for target gene B4GALT3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target SLC44A5' Retry (8/10). Failure caused by ValueError("Regression for target gene SLC44A5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target KIAA2013' Retry (10/10). Failure caused by ValueError("Regression for target gene KIAA2013 failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fa1e0>, (<function infer_partial_network at 0x7f22387e22f0>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

'infer_data failed for target PI4KB' Retry (1/10). Failure caused by ValueError("Regression for target gene PI4KB failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target LRRC38' Retry (9/10). Failure caused by ValueError("Regression for target gene LRRC38 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CSMD2' Retry (10/10). Failure caused by ValueError("Regression for target gene CSMD2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PIK3CD' Retry (9/10). Failure caused by ValueError("Regression for target gene PIK3CD failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fa158>, (<function infer_partial_network at 0x7f22387e2268>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

'infer_data failed for target GIPC2' Retry (10/10). Failure caused by ValueError("Regression for target gene GIPC2 failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fa158>, (<function infer_partial_network at 0x7f22387e2268>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

'infer_data failed for target TMCO4' Retry (8/10). Failure caused by ValueError("Regression for target gene TMCO4 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CNTN2' Retry (2/10). Failure caused by ValueError("Regression for target gene CNTN2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AC093158.1' Retry (9/10). Failure caused by ValueError("Regression for target gene AC093158.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL031429.1' Retry (8/10). Failure caused by ValueError("Regression for target gene AL031429.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target C1orf109' Retry (8/10). Failure caused by ValueError("Regression for target gene C1orf109 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target MRPS14' Retry (1/10). Failure caused by ValueError("Regression for target gene MRPS14 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL391832.2' Retry (2/10). Failure caused by ValueError("Regression for target gene AL391832.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target BMP8A' Retry (9/10). Failure caused by ValueError("Regression for target gene BMP8A failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AK5' Retry (9/10). Failure caused by ValueError("Regression for target gene AK5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PI4KB' Retry (2/10). Failure caused by ValueError("Regression for target gene PI4KB failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target LRRC38' Retry (10/10). Failure caused by ValueError("Regression for target gene LRRC38 failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fa158>, (<function infer_partial_network at 0x7f22387e2268>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

'infer_data failed for target PIK3CD' Retry (10/10). Failure caused by ValueError("Regression for target gene PIK3CD failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target B4GALT3' Retry (2/10). Failure caused by ValueError("Regression for target gene B4GALT3 failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fa158>, (<function infer_partial_network at 0x7f22387e2268>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

'infer_data failed for target TRMT1L' Retry (1/10). Failure caused by ValueError("Regression for target gene TRMT1L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TMCO4' Retry (9/10). Failure caused by ValueError("Regression for target gene TMCO4 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target DUSP12' Retry (1/10). Failure caused by ValueError("Regression for target gene DUSP12 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CNTN2' Retry (3/10). Failure caused by ValueError("Regression for target gene CNTN2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target SLC44A5' Retry (9/10). Failure caused by ValueError("Regression for target gene SLC44A5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL031429.1' Retry (9/10). Failure caused by ValueError("Regression for target gene AL031429.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target FBXO6' Retry (10/10). Failure caused by ValueError("Regression for target gene FBXO6 failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fa158>, (<function infer_partial_network at 0x7f22387e2268>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

'infer_data failed for target AC093158.1' Retry (10/10). Failure caused by ValueError("Regression for target gene AC093158.1 failed. Cause ValueError('buffer source array is read-only')."). distributed.worker - WARNING - Compute Failed Function: execute_task args: ((<function check_meta at 0x7f22488fb158>, (<function infer_partial_network at 0x7f22387e2268>, 'GBM', (<class 'dict'>, [['learning_rate', 0.01], ['n_estimators', 5000], ['max_features', 0.1], ['subsample', 0.9]]), array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 1., 1., ..., 0., 0., 0.], ..., [1., 0., 1., ..., 1., 0., 0.], [2., 0., 0., ..., 0., 0., 0.], [0., 0., 1., ..., 0., 0., 0.]], dtype=float32), ['NOC2L', 'HES4', 'CPTP', 'MORN1', 'HES5', 'PRDM16', 'TP73', 'HES2', 'ZBTB48', 'VAMP3', 'ENO1', 'AGMAT', 'ZBTB17', 'HP1BP3', 'ZBTB40', 'LUZP1', 'ZNF436', 'ASAP3', 'E2F2', 'RUNX3', 'LIN28A', 'GMEB1', 'ZCCHC17', 'HDAC1', 'ZBTB8B', 'ZNF362', 'ZSCAN20', 'SFPQ', 'TFAP2E', 'MTF1', 'POU3F1', 'HEYL', 'MYCL', 'RLF', 'SMAP2', 'ZFP69B', 'ZFP69', 'EXO5', 'ZNF684', 'NFYC', 'SCMH1', 'FOXO6', 'HIVEP3', 'FOXJ3', 'YBX1', 'ZNF691', 'KDM4A', 'KLF17', 'DMAP1', 'AKR1A1', 'DMBX1', 'FOXE3', 'FOXD2', 'DMRTA2', 'MAGOH', 'GLIS1', 'SSBP3', 'PRKAA2', kwargs: {} Exception: ValueError('Metadata mismatch found in from_delayed.\n\nExpected partition of type DataFrame but got NoneType')

not shutting down client, client was created externally finished 'infer_data failed for target DENND2D' Retry (1/10). Failure caused by ValueError("Regression for target gene DENND2D failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target MRPS14' Retry (2/10). Failure caused by ValueError("Regression for target gene MRPS14 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target C1orf109' Retry (9/10). Failure caused by ValueError("Regression for target gene C1orf109 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL391832.2' Retry (3/10). Failure caused by ValueError("Regression for target gene AL391832.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PI4KB' Retry (3/10). Failure caused by ValueError("Regression for target gene PI4KB failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TRMT1L' Retry (2/10). Failure caused by ValueError("Regression for target gene TRMT1L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ARHGEF11' Retry (1/10). Failure caused by ValueError("Regression for target gene ARHGEF11 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target BMP8A' Retry (10/10). Failure caused by ValueError("Regression for target gene BMP8A failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target DUSP12' Retry (2/10). Failure caused by ValueError("Regression for target gene DUSP12 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CNTN2' Retry (4/10). Failure caused by ValueError("Regression for target gene CNTN2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target B4GALT3' Retry (3/10). Failure caused by ValueError("Regression for target gene B4GALT3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TMCO4' Retry (10/10). Failure caused by ValueError("Regression for target gene TMCO4 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL031429.1' Retry (10/10). Failure caused by ValueError("Regression for target gene AL031429.1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AK5' Retry (10/10). Failure caused by ValueError("Regression for target gene AK5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target UAP1' Retry (1/10). Failure caused by ValueError("Regression for target gene UAP1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target SLC44A5' Retry (10/10). Failure caused by ValueError("Regression for target gene SLC44A5 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ANKRD45' Retry (1/10). Failure caused by ValueError("Regression for target gene ANKRD45 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL391832.2' Retry (4/10). Failure caused by ValueError("Regression for target gene AL391832.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target DENND2D' Retry (2/10). Failure caused by ValueError("Regression for target gene DENND2D failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target MRPS14' Retry (3/10). Failure caused by ValueError("Regression for target gene MRPS14 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PI4KB' Retry (4/10). Failure caused by ValueError("Regression for target gene PI4KB failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CNTN2' Retry (5/10). Failure caused by ValueError("Regression for target gene CNTN2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target DUSP12' Retry (3/10). Failure caused by ValueError("Regression for target gene DUSP12 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TRMT1L' Retry (3/10). Failure caused by ValueError("Regression for target gene TRMT1L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target C1orf109' Retry (10/10). Failure caused by ValueError("Regression for target gene C1orf109 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ARHGEF11' Retry (2/10). Failure caused by ValueError("Regression for target gene ARHGEF11 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target B4GALT3' Retry (4/10). Failure caused by ValueError("Regression for target gene B4GALT3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target UAP1' Retry (2/10). Failure caused by ValueError("Regression for target gene UAP1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ANKRD45' Retry (2/10). Failure caused by ValueError("Regression for target gene ANKRD45 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL391832.2' Retry (5/10). Failure caused by ValueError("Regression for target gene AL391832.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target DENND2D' Retry (3/10). Failure caused by ValueError("Regression for target gene DENND2D failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target MRPS14' Retry (4/10). Failure caused by ValueError("Regression for target gene MRPS14 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PI4KB' Retry (5/10). Failure caused by ValueError("Regression for target gene PI4KB failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CNTN2' Retry (6/10). Failure caused by ValueError("Regression for target gene CNTN2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target TRMT1L' Retry (4/10). Failure caused by ValueError("Regression for target gene TRMT1L failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ARHGEF11' Retry (3/10). Failure caused by ValueError("Regression for target gene ARHGEF11 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target B4GALT3' Retry (5/10). Failure caused by ValueError("Regression for target gene B4GALT3 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target DUSP12' Retry (4/10). Failure caused by ValueError("Regression for target gene DUSP12 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target ANKRD45' Retry (3/10). Failure caused by ValueError("Regression for target gene ANKRD45 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target AL391832.2' Retry (6/10). Failure caused by ValueError("Regression for target gene AL391832.2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target UAP1' Retry (3/10). Failure caused by ValueError("Regression for target gene UAP1 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target DENND2D' Retry (4/10). Failure caused by ValueError("Regression for target gene DENND2D failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target CNTN2' Retry (7/10). Failure caused by ValueError("Regression for target gene CNTN2 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target MRPS14' Retry (5/10). Failure caused by ValueError("Regression for target gene MRPS14 failed. Cause ValueError('buffer source array is read-only')."). 'infer_data failed for target PI4KB' Retry (6/10). Failure caused by ValueError("Regression for target gene PI4KB failed. Cause ValueError('buffer source array is read-only')."). Traceback (most recent call last): File "/home/kbaral/anaconda3/bin/pyscenic", line 10, in sys.exit(main()) File "/home/kbaral/anaconda3/lib/python3.7/site-packages/pyscenic/cli/pyscenic.py", line 420, in main args.func(args) File "/home/kbaral/anaconda3/lib/python3.7/site-packages/pyscenic/cli/pyscenic.py", line 72, in find_adjacencies_command network = method(expression_data=ex_mtx, tf_names=tf_names, verbose=True, client_or_address=client, seed=args.seed) File "/home/kbaral/anaconda3/lib/python3.7/site-packages/arboreto/algo.py", line 41, in grnboost2 early_stop_window_length=early_stop_window_length, limit=limit, seed=seed, verbose=verbose) File "/home/kbaral/anaconda3/lib/python3.7/site-packages/arboreto/algo.py", line 135, in diy .compute(graph, sync=True) \ File "/home/kbaral/anaconda3/lib/python3.7/site-packages/distributed/client.py", line 2758, in compute result = self.gather(futures) File "/home/kbaral/anaconda3/lib/python3.7/site-packages/distributed/client.py", line 1822, in gather asynchronous=asynchronous, File "/home/kbaral/anaconda3/lib/python3.7/site-packages/distributed/client.py", line 753, in sync return sync(self.loop, func, *args, *kwargs) File "/home/kbaral/anaconda3/lib/python3.7/site-packages/distributed/utils.py", line 331, in sync six.reraise(error[0]) File "/home/kbaral/anaconda3/lib/python3.7/site-packages/six.py", line 693, in reraise raise value File "/home/kbaral/anaconda3/lib/python3.7/site-packages/distributed/utils.py", line 316, in f result[0] = yield future File "/home/kbaral/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 735, in run value = future.result() File "/home/kbaral/anaconda3/lib/python3.7/site-packages/tornado/gen.py", line 742, in run yielded = self.gen.throw(*exc_info) # type: ignore File "/home/kbaral/anaconda3/lib/python3.7/site-packages/distributed/client.py", line 1653, in _gather six.reraise(type(exception), exception, traceback) File "/home/kbaral/anaconda3/lib/python3.7/site-packages/six.py", line 692, in reraise raise value.with_traceback(tb) File "/home/kbaral/anaconda3/lib/python3.7/site-packages/dask/dataframe/utils.py", line 521, in check_meta errmsg)) ValueError: Metadata mismatch found in from_delayed.

Expected partition of type DataFrame but got NoneType distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-13, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-8, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-2, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-12, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-16, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-7, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-20, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-1, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-15, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-19, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-10, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-14, started daemon)> distributed.process - WARNING - reaping stray process <ForkServerProcess(ForkServerProcess-18, started daemon)> distributed.nanny - WARNING - Worker process 155392 was killed by signal 15 distributed.nanny - WARNING - Worker process 155366 was killed by signal 15 distributed.nanny - WARNING - Worker process 155390 was killed by signal 15 distributed.nanny - WARNING - Worker process 155380 was killed by unknown signal distributed.nanny - WARNING - Worker process 155368 was killed by unknown signal distributed.nanny - WARNING - Worker process 155378 was killed by signal 15 distributed.nanny - WARNING - Worker process 155402 was killed by signal 15 distributed.nanny - WARNING - Worker process 155394 was killed by signal 15 distributed.nanny - WARNING - Worker process 155384 was killed by signal 15 distributed.nanny - WARNING - Worker process 155400 was killed by signal 15 distributed.nanny - WARNING - Worker process 155388 was killed by unknown signal distributed.nanny - WARNING - Worker process 155404 was killed by signal 15 distributed.nanny - WARNING - Worker process 155396 was killed by unknown signal

JBreunig commented 4 years ago

No, I haven't seen those errors. I did have to play with the num_workers argument but those warning were different.

I adjusted my code from this: https://github.com/aertslab/pySCENIC/blob/master/notebooks/pySCENIC%20-%20Integration%20with%20scanpy.ipynb

But I've now used it on over a dozen datasets without issue (10X and Smart-seq v4).

I might recommend just running this tutorial to try to determine whether it is version issues, data format, or something else.

KabitaBaral1 commented 4 years ago

Thank you very much @JBreunig . I tweaked it and got it to work, except the part where I have to put the outputs from GRN, AUCell and regulon together into a loom file. The problem is that I ran my auc in CLI so I don't have it as matrix, and when I try to run this excerpt of code:

add_scenic_metadata(adata, auc_mtx, regulons)

I keep running into AssertionError: blank assertion error. Do you have any other idea on creating loom file from these three files to compile them?

Thank you . I appreciate your help.

JBreunig commented 4 years ago

The particular step "add_scenic_metadata(adata, auc_mtx, regulons)" is only adding the metadata to the anndata structure. The loom output is farther down (export2loom) but I don't think that's the issue here.

The more likely issue is that I think that there were some variable name and folder name ambiguities in that particular notebook that you have to adjust/correct or maybe your matrices aren't matching up. For example if I recollect correctly with auc_mtx = aucell(exp_mtx, regulons, num_workers=20), I don't think that exp_mtx is loaded and so you have to do that elsewhere.

For the second potential issue, did you check that the size of your auc_mtx and exp_mtx correspond in the appropriate dimension (i.e. cell number only as genes vs. regulons will be different between the two)? auc_mtx.shape exp_mtx.shape

I recently was unable to perform this step on a 300,000K cell dataset, leading to me reinstalling anaconda with no luck--still troubleshooting that. But otherwise I haven't had an error as long as my inputs (adata[and thus exp_mtx], and auc_mtx) matched up. If I make an upstream change in adata that doesn't correspond to the raw matrix I'd passed to pySCENIC, it will cause issues because of size mismatches.

KabitaBaral1 commented 4 years ago

Hi, Thank you for the help. Yes, I somehow fixed it but I get another error while trying to write into a loom file using the command: export2loom(df_tpm.T, regulons, LOOM_FNAME_1, cell_annotations=adata_1.obs['cell'].to_dict(), tree_structure=(), title='Schwann_Cells', compress=True)

I keep getting this error: Object dtype dtype('O') has no native HDF5 equivalent

I saw this error in GitHUb but found no solution to it. Did you ever run into this error? Do you have any idea on how to fix it?

Thank you. Much appreciated. And sorry I keep bugging you.

Thank you

JBreunig commented 4 years ago

No worries...happy to give what little assistance I can offer ;)

Unfortunately, I haven't had time to troubleshoot that function. I tried once it didn't include the features that I wanted in SCOPE because I didn't have them in the appropriate format to add to the LOOM. It may have been that same error and it caused me to drop some of the data. (i.e., if you remove the cell_annotations, or tree structure argument, does it complete successfully? If so, something in one of those is incompatible with the HDF5 format.) https://github.com/linnarsson-lab/loompy/issues/12

I general, I am just saving things as an h5ad file.

KabitaBaral1 commented 4 years ago

I see, Thank you. I tried without cell_annotations, and it works but it does not let me open the loom file in R. My plan was to open loom file in R for data analysis. Did you ever get this warning for regulons?:

Regulon name does not seem to be compatible with SCOPE. It should include a space to allow selection of the TF. Please run: regulons = [r.rename(r.name.replace('(+)',' ('+str(len(r))+'g)')) for r in regulons] or: regulons = [r.rename(r.name.replace('(',' (')) for r in regulons]

I believe that this might be the issue here. Also, if you never create the loom file, how do you analyze your output? And, do you open h5ad file in R and perform further analysis?

I think that's my main concerns. (i) if my regulons are in correct format. I followed the vignette you sent link to for regulons. (ii) where and how do I perform data analysis and (iii) how to I compile them all together.

Thank you again.

JBreunig commented 4 years ago

I'm mainly looking at my data in Scanpy. An alternate way I was going to try to make the loom was to start at [65] in this notebook: https://github.com/aertslab/SCENICprotocol/blob/master/notebooks/PBMC10k_SCENIC-protocol-CLI.ipynb

I haven't had a chance yet...good luck!

Pentayouth commented 4 years ago

What worked for me was to rename the colnames of the GRNBoost output .tsv required by the downstream wrapper runSCENIC_1_coexNetwork2modules() and save the file in the respective scenic int/ directory.

e.g.: GRNBoost_output <- read.delim("~dir_to_GRNboost_tsv_file/ex_5000SC_network.tsv", header=FALSE) colnames(GRNBoost_output) <- c("TF","Target","weight") saveRDS(GRNBoost_output, file="int/1.4_GENIE3_linkList.Rds")

I believe the "header" parameter for the function "read.delim" should be "TRUE". Using FALSE will cause an error because the header would be mistakenly treated as the first row.

JABioinf commented 1 year ago

Hi, I have been running pySCENIC using singularity and got the 3 main important files: auc_mtx.csv, adjacencies.tsv, regulons.csv. Was there any updated method to import those outputs into SCENIC in R?
Is there a way to save them to be read directly with the ScenicOptions class in R? source: https://pyscenic.readthedocs.io/en/latest/installation.html Thank you!

jpezoldt commented 1 year ago

I haven't used the package since 2018 so am not aware of any updates. Stein's team should be on top of this.Maybe you already figured it out. Sorry for the late reply and the zero input. Regardless, have a nice Easter. Joern

 

Jörn Pezoldt Mobil: +49-1577-3818562 @.***

Am Donnerstag, 16. März 2023 um 02:12:48 MEZ hat J_A ***@***.***> Folgendes geschrieben:

Hi, I have been running pySCENIC using singularity and got the 3 main important files: auc_mtx.csv, adjacencies.tsv, regulons.csv. Was there any updated method to import those outputs into SCENIC in R? Is there a way to save them to be read directly with the ScenicOptions class in R? source: https://pyscenic.readthedocs.io/en/latest/installation.html Thank you!

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

xinpengdun commented 1 year ago

the easist way is to downlaod 1.4_GENIE3_linkList.Rds, and move it to your int fold: you can download 1.4_GENIE3_linkList.Rds from https://scenic.aertslab.org/scenic_paper/examples/SCENIC_MouseBrain/int/