zhangdong360
commented
4 months ago My code:
#!/bin/bash
#SBATCH -o output/pyscenic_hsc_sev.out
#SBATCH -e output/pyscenic_hsc_sev.err
#SBATCH --partition=compute
#SBATCH -J scenic_HSC_SEV
#SBATCH --nodes=1
#SBATCH -n 30
# This is for fastp protocol
#conda activate scanpy
# human
#f_db_names="/share/home/zhangd/tools/database/cistarget/cisTarget_databases/homo_sapiens/hg38/refseq_r80/mc_v10_clust/gene_based/hg38_500bp_up_100bp_down_full_tx_v10_clust.genes_vs_motifs.rankings.feather"
#f_motif_path="/share/home/zhangd/tools/database/cistarget/Motif2TF/motifs-v10nr_clust-nr.hgnc-m0.001-o0.0.tbl"
#f_tf_list="/share/home/zhangd/project/python_project/pySCENIC/allTFs_hg38.txt"
# mouse
f_db_names="/home/zhangdong_2/database/cistarget/cisTarget_databases/mus_musculus/mm10/refseq_r80/mc_v10_clust/mm10_500bp_up_100bp_down_full_tx_v10_clust.genes_vs_motifs.rankings.feather"
f_motif_path="/home/zhangdong_2/database/cistarget/Motif2TF/motifs-v10nr_clust-nr.mgi-m0.001-o0.0.tbl"
f_tf_list="/home/zhangdong_2/database/cistarget/TF_lists/allTFs_mm.txt"
# data input
dir_result="/home/zhangdong_2/project/pySCENIC/03_result/HSC_SEV/"
input_loom="/home/zhangdong_2/project/pySCENIC/01_data/HSC_SEV.loom"
# step1
echo "Step 1 pyscenic grn start"
nohup pyscenic grn ${input_loom} ${f_tf_list} \
--seed 21 \
--num_workers 16 \
--method grnboost2 \
--output ${dir_result}/step_1_fibo_grn.tsv >step1.out 2>&1 &
echo "Step 1 pyscenic grn finish"
echo "Step 2 pyscenic ctx start"
nohup pyscenic ctx ${dir_result}/step_1_fibo_grn.tsv \
${f_db_names} \
--annotations_fname ${f_motif_path} \
--expression_mtx_fname ${input_loom} \
--output ${dir_result}/step_2_reg.csv \
--mask_dropouts \
--num_workers 16 >step2.out 2>&1 &
echo "Step 2 pyscenic ctx finish"
echo "Step 3 pyscenic aucell start"
pyscenic aucell \
${input_loom} \
${dir_result}/step_2_reg.csv \
--seed 21 \
--output ${dir_result}/step_3_aucell.csv \
--num_workers 16 >step_3.out 2>&1 &
echo "All finish"I tried to use slurm distribution to compute nodes and directly in local bash operation, the result is the same.
At the same time I run this code on the other operation platform, I found warning almost consistent, all point to the ndarray - a612cf0abd06497fa68e9db39636fedb.
May be some help to found the problem?
ghuls
JustMoveOnnn
Flu09
sherinesaber
lidoctor
Describe the bug When I run pySCENIC, I often encounter disturbing warnings. I checked the problem may be associated with me this question. https://github.com/aertslab/pySCENIC/issues/482 But I'm not using port 8787. On the other hand, I don't often encounter this warning on the HPC where I have Rstudio server installed, and I don't think it has anything to do with it. I think the problem might be with dask, but I'm not well versed in it. On the other hand, the lack of output, which makes me cannot judge whether I need to run the program. As mentioned above, re-running the program will most likely encounter warning again. In addition, I have tried arboreto_with_multiprocessing.py, but it was too inefficient, I tested it on small samples, and it was nearly twice as slow as pySCENIC for the same number of CPU cores. I don't think that's acceptable in a large sample. It took me too much energy in to run the program, I have to sample cut to my data size, but I don't think this is a long-term solution.
Expected behavior I didn't find a clear reproduction. But I find it often will appear in my after a run.