Pyscenic ctx Feather Error

[Sayyam-Shah]

Hello,

Thank you for making the amazing tool. I ran grn and got my adj.mtx file. However, I'm getting errors upon reading in the ranking dataset with ctx. I ran the below command.

pyscenic ctx adjfem70wi.csv \ hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.feather hg38refseq-r8010kb_up_and_down_tss.mc9nr.feather\ --annotations_fname motifs-v9-nr.hgnc-m0.001-o0.0.tbl \ --expression_mtx_fname fem70exprmatrix.csv \ --output regfem70.csv \ --mask_dropouts \ --num_workers 5

Traceback (most recent call last): File "/cluster/home/.local/bin/pyscenic", line 8, in sys.exit(main()) File "/cluster/home/.local/lib/python3.7/site-packages/pyscenic/cli/pyscenic.py", line 677, in main args.func(args) File "/cluster/home/.local/lib/python3.7/site-packages/pyscenic/cli/pyscenic.py", line 215, in prune_targets_command dbs = _load_dbs(args.database_fname) File "/cluster/home/.local/lib/python3.7/site-packages/pyscenic/cli/pyscenic.py", line 176, in _load_dbs return [opendb(fname=fname.name, name=get_name(fname.name)) for fname in fnames] File "/cluster/home/.local/lib/python3.7/site-packages/pyscenic/cli/pyscenic.py", line 176, in return [opendb(fname=fname.name, name=get_name(fname.name)) for fname in fnames] File "/cluster/home/.local/lib/python3.7/site-packages/ctxcore/rnkdb.py", line 180, in opendb return FeatherRankingDatabase(fname, name=name) File "/cluster/home/.local/lib/python3.7/site-packages/ctxcore/rnkdb.py", line 110, in init ct_db_filename=self._fname, engine="pyarrow" File "/cluster/home/.local/lib/python3.7/site-packages/ctxcore/ctdb.py", line 175, in init_ct_db f'"{ct_db_filename}" is a cisTarget Feather database in Feather v1 format, which is not supported ' ValueError: "hg38refseq-r80500bp_up_and_100bp_down_tss.mc9nr.feather" is a cisTarget Feather database in Feather v1 format, which is not supported anymore. Convert them with "convert_cistarget_databases_v1_to_v2.py" (https://github.com/aertslab/create_cisTarget_databases/) to Feather v2 format.

I then converted the ranking database to feather v2 as suggested and got the below error.

Traceback (most recent call last): File "/cluster/home/.local/bin/pyscenic", line 8, in sys.exit(main()) File "/cluster/home/.local/lib/python3.7/site-packages/pyscenic/cli/pyscenic.py", line 677, in main args.func(args) File "/cluster/home/.local/lib/python3.7/site-packages/pyscenic/cli/pyscenic.py", line 215, in prune_targets_command dbs = _load_dbs(args.database_fname) File "/cluster/home/.local/lib/python3.7/site-packages/pyscenic/cli/pyscenic.py", line 176, in _load_dbs return [opendb(fname=fname.name, name=get_name(fname.name)) for fname in fnames] File "/cluster/home/.local/lib/python3.7/site-packages/pyscenic/cli/pyscenic.py", line 176, in return [opendb(fname=fname.name, name=get_name(fname.name)) for fname in fnames] File "/cluster/home/.local/lib/python3.7/site-packages/ctxcore/rnkdb.py", line 180, in opendb return FeatherRankingDatabase(fname, name=name) File "/cluster/home/.local/lib/python3.7/site-packages/ctxcore/rnkdb.py", line 110, in init ct_db_filename=self._fname, engine="pyarrow" File "/cluster/home/.local/lib/python3.7/site-packages/ctxcore/ctdb.py", line 293, in init_ct_db ) = get_ct_db_type_from_ct_db_filename(ct_db_filename) File "/cluster/home/.local/lib/python3.7/site-packages/ctxcore/ctdb.py", line 104, in get_ct_db_type_from_ct_db_filename f'cisTarget database filename "{ct_db_filename}" does not end with ' ValueError: cisTarget database filename "hg38refseq-r80500bp_up_and_100bp_down_tssv2.mc9nr.feather" does not end with ".((motifs|tracks)vs(regions|genes)|(regions|genes)vs(motifs|tracks)).(scores|rankings).feather".

How may I fix this? I got the dataset from the following link: https://resources.aertslab.org/cistarget/

Thank you!

[pillar-github]

i have the same problem!

[cbravo93]

Hi!

You can just rename the feather v2 databases from hg38__refseq-r80__500bp_up_and_100bp_down_tssv2.mc9nr.feather to hg38__refseq-r80__500bp_up_and_100bp_down_tss.genes_vs_motifs.rankings.feather and it will work. We will make make the databases compatible with the new pySCENIC/ctxcore versions shortly (feather v2 and ending in genes_vs_motifs.rankings.feather) in https://resources.aertslab.org/cistarget/ ; but this will work with your current dbs without issues.

Cheers!

Carmen

[pillar-github]

it works!! thanks a lot!

[zyb1984]

Does any other reference information need to be updated, such as the motifs.tbl and allTFs_hg38.txt, maybe in other words, how I can choose a suitable list (including *.feather, motif.tbl and TF_list.txt) for this analysis?

[JoGraesslin]

Hello. I want to try out scenic in zebrafish, and I want to use the CIS-BP database file that has been recently used by Liu et al. 2022 https://ftp.cngb.org/pub/SciRAID/stomics/STDS0000057/cisTarget/ as a reference. After updating I got the "does not end with ".((motifs|tracks)vs(regions|genes)|(regions|genes)vs(motifs|tracks)).(scores|rankings).feather"." error message, so I changed the file names accordingly. Now I get another error message: ValueError: cisTarget database "zebrafish_cisTarget/zebrafish_v2_tss.genes_vs_motifs.rankings.feather" claims to contain motifs based on the filename, but contains tracks. Any idea what is going on? Help would be highly appreciated! Best, Jo

[pennell01]

Hi!

You can just rename the feather v2 databases from hg38__refseq-r80__500bp_up_and_100bp_down_tssv2.mc9nr.feather to hg38__refseq-r80__500bp_up_and_100bp_down_tss.genes_vs_motifs.rankings.feather and it will work. We will make make the databases compatible with the new pySCENIC/ctxcore versions shortly (feather v2 and ending in genes_vs_motifs.rankings.feather) in https://resources.aertslab.org/cistarget/ ; but this will work with your current dbs without issues.

Cheers!

Carmen

Hello! My dear friend, Would you mind sharing your feather v2 databases to me as I had difficulties to convert the version, thanks a lot!

[CWYuan08]

@JoGraesslin Hi, I have the same issue, do you have any solution for this? Many thanks!!