search

aertslab / scenicplus

SCENIC+ is a python package to build gene regulatory networks (GRNs) using combined or separate single-cell gene expression (scRNA-seq) and single-cell chromatin accessibility (scATAC-seq) data.
Other
165 stars 27 forks source link

DEM analysis error: 'dict_keys' object is not subscriptable #237

Closed jhong26 closed 9 months ago

jhong26 commented 9 months ago

Hello,

I am trying to run scenicplus in one cell type comparing two conditions. I'd like to keep the default thresholds for consistency across other cell types. Condition1 has zero differential features with default thresholds and so I added some code suggested in another post. However, with the run_pycistarget step, I get the error 'dict_keys' object is not subscriptable at the DEM analysis step. Any input on how to fix this error would be much appreciated. Thanks so much.

markers_dict = find_diff_features(cistopic_obj, imputed_acc_obj, variable='Status', var_features=variable_regions, adjpval_thr=0.05, log2fc_thr=np.log2(1.5))

for DAR in markers_dict.keys(): regions = markers_dict[DAR].index[markers_dict[DAR].index.str.startswith('chr')] #only keep regions on known chromosomes if len(regions) > 0: region_sets['DARs'][DAR] = pr.PyRanges(region_names_to_coordinates(regions))

Below is the output:

2023-10-02 12:06:53,071 cisTopic INFO Imputing region accessibility 2023-10-02 12:06:53,071 cisTopic INFO Impute region accessibility for regions 0-20000 ... 2023-10-02 12:06:55,514 cisTopic INFO Impute region accessibility for regions 160000-180000 2023-10-02 12:06:55,528 cisTopic INFO Done! 2023-10-02 12:06:55,529 cisTopic INFO Normalizing imputed data 2023-10-02 12:06:56,832 cisTopic INFO Done! 2023-10-02 12:06:56,833 cisTopic INFO Calculating mean 2023-10-02 12:06:57,068 cisTopic INFO Calculating variance 2023-10-02 12:07:00,891 cisTopic INFO Done! 2023-10-02 12:07:01,167 cisTopic INFO Subsetting data for Condition1 (445 of 1312) 2023-10-02 12:07:08,568 cisTopic INFO Computing p-value for Condition1 2023-10-02 12:07:26,022 cisTopic INFO Computing log2FC for Condition1 2023-10-02 12:07:27,684 cisTopic INFO Condition1 done! 2023-10-02 12:07:27,686 cisTopic INFO Subsetting data for Condition2 (867 of 1312) 2023-10-02 12:07:27,703 cisTopic INFO Computing p-value for Condition2 2023-10-02 12:07:45,218 cisTopic INFO Computing log2FC for Condition2 2023-10-02 12:07:45,249 cisTopic INFO Condition2 done! topics_otsu: dict_keys(['Topic1', 'Topic2', 'Topic3', 'Topic4', 'Topic5', 'Topic6', 'Topic7', 'Topic8', 'Topic9', 'Topic10', 'Topic11', 'Topic12', 'Topic13', 'Topic14', 'Topic15', 'Topic16', 'Topic17', 'Topic18', 'Topic19', 'Topic20', 'Topic21', 'Topic22', 'Topic23', 'Topic24', 'Topic25', 'Topic26', 'Topic27', 'Topic28', 'Topic29', 'Topic30', 'Topic31', 'Topic32', 'Topic33', 'Topic34', 'Topic35', 'Topic36', 'Topic37', 'Topic38', 'Topic39', 'Topic40', 'Topic41', 'Topic42', 'Topic43', 'Topic44', 'Topic45', 'Topic46', 'Topic47', 'Topic48', 'Topic49', 'Topic50', 'Topic51', 'Topic52', 'Topic53', 'Topic54', 'Topic55', 'Topic56', 'Topic57', 'Topic58', 'Topic59', 'Topic60', 'Topic61', 'Topic62', 'Topic63', 'Topic64', 'Topic65', 'Topic66', 'Topic67', 'Topic68', 'Topic69', 'Topic70']) topics_top_3: dict_keys(['Topic1', 'Topic2', 'Topic3', 'Topic4', 'Topic5', 'Topic6', 'Topic7', 'Topic8', 'Topic9', 'Topic10', 'Topic11', 'Topic12', 'Topic13', 'Topic14', 'Topic15', 'Topic16', 'Topic17', 'Topic18', 'Topic19', 'Topic20', 'Topic21', 'Topic22', 'Topic23', 'Topic24', 'Topic25', 'Topic26', 'Topic27', 'Topic28', 'Topic29', 'Topic30', 'Topic31', 'Topic32', 'Topic33', 'Topic34', 'Topic35', 'Topic36', 'Topic37', 'Topic38', 'Topic39', 'Topic40', 'Topic41', 'Topic42', 'Topic43', 'Topic44', 'Topic45', 'Topic46', 'Topic47', 'Topic48', 'Topic49', 'Topic50', 'Topic51', 'Topic52', 'Topic53', 'Topic54', 'Topic55', 'Topic56', 'Topic57', 'Topic58', 'Topic59', 'Topic60', 'Topic61', 'Topic62', 'Topic63', 'Topic64', 'Topic65', 'Topic66', 'Topic67', 'Topic68', 'Topic69', 'Topic70']) DARs: dict_keys(['Condition2']) 2023-10-02 12:07:52,702 pycisTarget_wrapper INFO mypath/scenicplus/ncM folder already exists. 2023-10-02 12:07:53,383 pycisTarget_wrapper INFO Loading cisTarget database for topics_otsu 2023-10-02 12:07:53,384 cisTarget INFO Reading cisTarget database 2023-10-02 12:24:19,012 pycisTarget_wrapper INFO Running cisTarget for topics_otsu 2023-10-02 12:24:19,013 cisTarget INFO Running cisTarget for Topic1 which has 23504 regions 2023-10-02 12:24:55,909 cisTarget INFO Annotating motifs for Topic1 2023-10-02 12:25:01,208 cisTarget INFO Getting cistromes for Topic1 ... 2023-10-02 12:59:16,396 cisTarget INFO Running cisTarget for Topic70 which has 35832 regions 2023-10-02 12:59:43,138 cisTarget INFO Annotating motifs for Topic70 2023-10-02 12:59:48,180 cisTarget INFO Getting cistromes for Topic70 2023-10-02 12:59:49,019 cisTarget INFO Done! 2023-10-02 12:59:49,026 pycisTarget_wrapper INFO mypath/scenicplus/ncM/CTX_topics_otsu_All folder already exists. 2023-10-02 13:37:45,506 pycisTarget_wrapper INFO Running cisTarget without promoters for topics_otsu 2023-10-02 13:39:13,579 cisTarget INFO Running cisTarget for Topic1 which has 3180 regions 2023-10-02 13:39:36,148 cisTarget INFO Annotating motifs for Topic1 2023-10-02 13:39:38,110 cisTarget INFO Getting cistromes for Topic1 ... 2023-10-02 14:07:23,501 cisTarget INFO Running cisTarget for Topic70 which has 14701 regions 2023-10-02 14:07:42,062 cisTarget INFO Annotating motifs for Topic70 2023-10-02 14:07:44,266 cisTarget INFO Getting cistromes for Topic70 2023-10-02 14:07:44,735 cisTarget INFO Done! 2023-10-02 14:07:44,741 pycisTarget_wrapper INFO mypath/scenicplus/ncM/CTX_topics_otsu_No_promoters folder already exists. 2023-10-02 14:16:01,949 pycisTarget_wrapper INFO Running DEM for topics_otsu 2023-10-02 14:16:01,950 DEM INFO Reading DEM database 2023-10-02 14:25:14,014 DEM INFO Creating contrast groups 2023-10-02 14:28:32,302 DEM INFO Computing DEM for Topic1 ... 2023-10-02 16:06:45,605 DEM INFO Computing DEM for Topic70 2023-10-02 16:08:08,748 DEM INFO Forming cistromes 2023-10-02 16:08:31,980 DEM INFO Done! 2023-10-02 16:08:31,995 pycisTarget_wrapper INFO mypath/scenicplus/ncM/DEM_topics_otsu_All folder already exists. 2023-10-02 16:10:53,428 pycisTarget_wrapper INFO Running DEM without promoters for topics_otsu 2023-10-02 16:12:24,671 DEM INFO Creating contrast groups 2023-10-02 16:13:08,233 DEM INFO Computing DEM for Topic1 ... 2023-10-02 16:39:54,625 DEM INFO Computing DEM for Topic70 2023-10-02 16:40:12,386 DEM INFO Forming cistromes 2023-10-02 16:40:33,747 DEM INFO Done! 2023-10-02 16:40:33,749 pycisTarget_wrapper INFO mypath/scenicplus/ncM/DEM_topics_otsu_No_promoters folder already exists. 2023-10-02 16:42:42,096 pycisTarget_wrapper INFO Loading cisTarget database for topics_top_3 2023-10-02 16:42:42,097 cisTarget INFO Reading cisTarget database 2023-10-02 16:48:14,798 pycisTarget_wrapper INFO Running cisTarget for topics_top_3 2023-10-02 16:48:14,799 cisTarget INFO Running cisTarget for Topic1 which has 13234 regions 2023-10-02 16:48:51,650 cisTarget INFO Annotating motifs for Topic1 2023-10-02 16:48:55,303 cisTarget INFO Getting cistromes for Topic1 ... 2023-10-02 17:20:29,029 cisTarget INFO Running cisTarget for Topic70 which has 9399 regions 2023-10-02 17:20:57,538 cisTarget INFO Annotating motifs for Topic70 2023-10-02 17:21:00,392 cisTarget INFO Getting cistromes for Topic70 2023-10-02 17:21:00,924 cisTarget INFO Done! 2023-10-02 17:21:00,925 pycisTarget_wrapper INFO mypath/scenicplus/ncM/CTX_topics_top_3_All folder already exists. 2023-10-02 17:35:09,751 pycisTarget_wrapper INFO Running cisTarget without promoters for topics_top_3 2023-10-02 17:36:15,785 cisTarget INFO Running cisTarget for Topic1 which has 1425 regions 2023-10-02 17:36:42,808 cisTarget INFO Annotating motifs for Topic1 2023-10-02 17:36:44,905 cisTarget INFO Getting cistromes for Topic1 ... 2023-10-02 18:03:32,134 cisTarget INFO Running cisTarget for Topic70 which has 3186 regions 2023-10-02 18:03:48,484 cisTarget INFO Annotating motifs for Topic70 2023-10-02 18:03:50,269 cisTarget INFO Getting cistromes for Topic70 2023-10-02 18:03:50,596 cisTarget INFO Done! 2023-10-02 18:03:50,598 pycisTarget_wrapper INFO mypath/scenicplus/ncM/CTX_topics_top_3_No_promoters folder already exists. 2023-10-02 18:07:47,155 pycisTarget_wrapper INFO Running DEM for topics_top_3 2023-10-02 18:07:47,156 DEM INFO Reading DEM database 2023-10-02 18:13:10,713 DEM INFO Creating contrast groups 2023-10-02 18:15:29,084 DEM INFO Computing DEM for Topic1 ... 2023-10-02 18:48:19,828 DEM INFO Computing DEM for Topic70 2023-10-02 18:48:56,752 DEM INFO Forming cistromes 2023-10-02 18:49:09,084 DEM INFO Done! 2023-10-02 18:49:09,085 pycisTarget_wrapper INFO mypath/scenicplus/ncM/DEM_topics_top_3_All folder already exists. 2023-10-02 18:49:57,598 pycisTarget_wrapper INFO Running DEM without promoters for topics_top_3 2023-10-02 18:51:01,465 DEM INFO Creating contrast groups 2023-10-02 18:51:21,104 DEM INFO Computing DEM for Topic1 ... 2023-10-02 19:04:52,948 DEM INFO Computing DEM for Topic70 2023-10-02 19:05:06,427 DEM INFO Forming cistromes 2023-10-02 19:05:20,009 DEM INFO Done! 2023-10-02 19:05:20,017 pycisTarget_wrapper INFO mypath/scenicplus/ncM/DEM_topics_top_3_No_promoters folder already exists. 2023-10-02 19:06:11,608 pycisTarget_wrapper INFO Loading cisTarget database for DARs 2023-10-02 19:06:11,609 cisTarget INFO Reading cisTarget database 2023-10-02 19:06:33,210 pycisTarget_wrapper INFO Running cisTarget for DARs 2023-10-02 19:06:33,210 cisTarget INFO Running cisTarget for Condition2 which has 6603 regions 2023-10-02 19:06:54,473 cisTarget INFO Annotating motifs for Condition2 2023-10-02 19:06:56,493 cisTarget INFO Getting cistromes for Condition2 2023-10-02 19:06:56,766 cisTarget INFO Done! 2023-10-02 19:06:56,767 pycisTarget_wrapper INFO mypath/scenicplus/ncM/CTX_DARs_All folder already exists. 2023-10-02 19:07:01,043 pycisTarget_wrapper INFO Running cisTarget without promoters for DARs 2023-10-02 19:07:01,488 cisTarget INFO Running cisTarget for Condition2 which has 5303 regions 2023-10-02 19:07:23,420 cisTarget INFO Annotating motifs for Condition2 2023-10-02 19:07:25,491 cisTarget INFO Getting cistromes for Condition2 2023-10-02 19:07:25,780 cisTarget INFO Done! 2023-10-02 19:07:25,781 pycisTarget_wrapper INFO mypath/scenicplus/ncM/CTX_DARs_No_promoters folder already exists. 2023-10-02 19:07:30,277 pycisTarget_wrapper INFO Running DEM for DARs 2023-10-02 19:07:30,278 DEM INFO Reading DEM database --------------------------------------------------------------------------- TypeError Traceback (most recent call last) File /u/project/xyang123/xyang123-NOBACKUP/jhong26/scRNAseq/PBMCMultiomeFinal/scenicplus/mydata/RunScenicPlus_ncM.py:97  92 # topics_otsu: dict_keys(['Topic1', 'Topic2', 'Topic3', 'Topic4', 'Topic5', 'Topic6', 'Topic7', 'Topic8', 'Topic9', 'Topic10', 'Topic11', 'Topic12', 'Topic13', 'Topic14', 'Topic15', 'Topic16', 'Topic17', 'Topic18', 'Topic19', 'Topic20', 'Topic21', 'Topic22', 'Topic23', 'Topic24', 'Topic25', 'Topic26', 'Topic27', 'Topic28', 'Topic29', 'Topic30', 'Topic31', 'Topic32', 'Topic33', 'Topic34', 'Topic35', 'Topic36', 'Topic37', 'Topic38', 'Topic39', 'Topic40', 'Topic41', 'Topic42', 'Topic43', 'Topic44', 'Topic45', 'Topic46', 'Topic47', 'Topic48', 'Topic49', 'Topic50', 'Topic51', 'Topic52', 'Topic53', 'Topic54', 'Topic55', 'Topic56', 'Topic57', 'Topic58', 'Topic59', 'Topic60', 'Topic61', 'Topic62', 'Topic63', 'Topic64', 'Topic65', 'Topic66', 'Topic67', 'Topic68', 'Topic69', 'Topic70'])  93 # topics_top_3: dict_keys(['Topic1', 'Topic2', 'Topic3', 'Topic4', 'Topic5', 'Topic6', 'Topic7', 'Topic8', 'Topic9', 'Topic10', 'Topic11', 'Topic12', 'Topic13', 'Topic14', 'Topic15', 'Topic16', 'Topic17', 'Topic18', 'Topic19', 'Topic20', 'Topic21', 'Topic22', 'Topic23', 'Topic24', 'Topic25', 'Topic26', 'Topic27', 'Topic28', 'Topic29', 'Topic30', 'Topic31', 'Topic32', 'Topic33', 'Topic34', 'Topic35', 'Topic36', 'Topic37', 'Topic38', 'Topic39', 'Topic40', 'Topic41', 'Topic42', 'Topic43', 'Topic44', 'Topic45', 'Topic46', 'Topic47', 'Topic48', 'Topic49', 'Topic50', 'Topic51', 'Topic52', 'Topic53', 'Topic54', 'Topic55', 'Topic56', 'Topic57', 'Topic58', 'Topic59', 'Topic60', 'Topic61', 'Topic62', 'Topic63', 'Topic64', 'Topic65', 'Topic66', 'Topic67', 'Topic68', 'Topic69', 'Topic70'])  94 # DARs: dict_keys(['Condition1', 'Condition2'])  96 rankings_db = 'mypath/scenicplus/ScreenDatabases/hg38_screen_v10_clust.regions_vs_motifs.rankings.feather' ---> 97 scores_db = 'mypath/scenicplus/ScreenDatabases/hg38_screen_v10_clust.regions_vs_motifs.scores.feather'  98 motif_annotation = 'mypath/scenicplus/motifs-v10nr_clust-nr.hgnc-m0.001-o0.0.tbl'  100 if not os.path.exists(os.path.join(work_dir, 'ncM')):

File ~/mypath/conda/scanpy/lib/python3.10/site-packages/scenicplus/wrappers/run_pycistarget.py:263, in run_pycistarget(region_sets, species, save_path, custom_annot, save_partial, ctx_db_path, dem_db_path, run_without_promoters, biomart_host, promoter_space, ctx_auc_threshold, ctx_nes_threshold, ctx_rank_threshold, dem_log2fc_thr, dem_motif_hit_thr, dem_max_bg_regions, annotation, motif_similarity_fdr, path_to_motif_annotations, annotation_version, n_cpu, _temp_dir, exclude_motifs, exclude_collection, *kwargs)  261 for col in exclude_collection:  262 dem_db.db_scores = dem_db.db_scores[~dem_db.dbscores.index.str.contains(col)] --> 263 menr['DEM'+key+'_All'] = DEM(dem_db = dem_db,  264  region_sets = regions,  265  log2fc_thr = dem_log2fc_thr,  266  motif_hit_thr = dem_motif_hit_thr,  267  max_bg_regions = dem_max_bg_regions,  268  specie = species,  269  genome_annotation = annot_dem,  270  promoter_space = promoter_space,  271  motif_annotation = annotation,  272  motif_similarity_fdr = motif_similarity_fdr,   273  path_to_motif_annotations = path_to_motif_annotations,  274  n_cpu = n_cpu,  275  annotation_version = annotation_version,  276  tmp_dir = save_path,  277  _temp_dir= _temp_dir,  278  *kwargs)  279 out_folder = os.path.join(savepath,'DEM'+key+'_All')  280 check_folder = os.path.isdir(out_folder)

File ~/mypath/conda/scanpy/lib/python3.10/site-packages/pycistarget/motif_enrichment_dem.py:319, in DEM.init(self, dem_db, region_sets, specie, subset_motifs, contrasts, name, max_bg_regions, adjpval_thr, log2fc_thr, mean_fg_thr, motif_hit_thr, n_cpu, fraction_overlap, cluster_buster_path, path_to_genome_fasta, path_to_motifs, genome_annotation, promoter_space, path_to_motif_annotations, annotation_version, motif_annotation, motif_similarity_fdr, orthologous_identity_threshold, tmp_dir, *kwargs)  317 self.cistromes = None  318 if dem_db is not None: --> 319 self.run(dem_db.db_scores, *kwargs)

File ~/mypath/conda/scanpy/lib/python3.10/site-packages/pycistarget/motif_enrichment_dem.py:349, in DEM.run(self, dem_db_scores, **kwargs)  347 contrasts_names=levels  348 else: --> 349 contrasts = [[self.region_sets.keys()[0]],['Shuffle']]  350 contrasts_names = [self.region_sets.keys()[0] + '_VS_Shuffle']
 352 elif isinstance(self.contrasts, list):

TypeError: 'dict_keys' object is not subscriptable

jhong26 commented 9 months ago

I found an explanation here: https://github.com/aertslab/pycistarget/issues/13