`ValueError: No objects to concatenate` during `motif_enrichment_dem` Execution

OjongTabi commented 3 months ago

Issue: `ValueError: No objects to concatenate` during `motif_enrichment_dem` Execution

scenicplus grn_inference motif_enrichment_dem \
  --region_set_folder /n/sci/SCI-004375-NYUDATA/Ojong/code/scenicplus_train_M21_1 \
  --dem_db_fname /n/sci/SCI-004375-NYUDATA/Ojong/code/scenicplus_train_M21_2/create_cisTarget_databases/M21.regions_vs_motifs.scores.feather \
  --output_fname_dem_result dem_results.hdf5 \
  --temp_dir /n/sci/SCI-004375-NYUDATA/Ojong/code/scenicplus_2024_14_8/tmp/ \
  --species Drosophila_melanogaster \
  --fraction_overlap_w_dem_database 0.4 \
  --max_bg_regions 500 \
  --genome_annotation genome_annotation.tsv \
  --balance_number_of_promoters \
  --promoter_space 1000 \
  --adjpval_thr 0.05 \
  --log2fc_thr 1.0 \
  --mean_fg_thr 0.0 \
  --motif_hit_thr 3.0 \
  --path_to_motif_annotations /n/sci/SCI-004375-NYUDATA/Ojong/code/aertslab_motif_colleciton/v10nr_clust_public/snapshots/motifs-v10-nr.flybase-m0.00001-o0.0.tbl \
  --annotation_version v10nr_clust \
  --motif_similarity_fdr 0.001 \
  --orthologous_identity_threshold 0.0 \
  --annotations_to_use Direct_annot Orthology_annot \
  --write_html \
  --output_fname_dem_html dem_results.html \
  --seed 666 \
  --n_cpu 20

Error Traceback:

joblib.externals.loky.process_executor._RemoteTraceback:
...
ValueError: No objects to concatenate

Description: While running the motif_enrichment_dem command, I encountered a ValueError: No objects to concatenate. The error suggests that there might be an issue with the input files or parameters, but all files appear to be correctly populated and formatted. The specific section of the code where the error occurs is within the get_foreground_and_background_regions function in pycistarget.

SeppeDeWinter commented 3 months ago

HI @OjongTabi

Can you calculate the overlap of your regions and the promoter regions:

promoter_annotation = genome_annotation.copy()
promoter_annotation["End"]   = promoter_annotation["Start"] + promoter_space
promoter_annotation["Start"] = promoter_annotation["Start"] - promoter_space

Best,

Seppe

Larrycpan commented 1 week ago

HI @OjongTabi

Can you calculate the overlap of your regions and the promoter regions:
promoter_annotation = genome_annotation.copy()
promoter_annotation["End"]   = promoter_annotation["Start"] + promoter_space
promoter_annotation["Start"] = promoter_annotation["Start"] - promoter_space
Best,

Seppe

I get into the same bug. I tested the overlap of one of my regions bed files (30,000+ regions) and the promoter regions (promoter_space = 1000), I found there were 20,000+ overlapped regions. So I cannot figure out the reason causing the bug.

aertslab / scenicplus