Mutex creation failed error

[AlmogAngel]

Dear developers,

When running the snakemake --use-conda test_immunedeconvz command I get the following error:

Rscript -e "library(methods); library(devtools); document()"
Updating immunedeconv documentation
Loading immunedeconv
Loading required package: EPIC
terminate called after throwing an instance of 'std::runtime_error'
  what():  Mutex creation failed
Aborted (core dumped)
make: *** [Makefile:18: roxygenize] Error 134

I have tried to remove EPIC from notebooks/config.R but it doesn't solve the problem.

Please let me know if you have an idea how to fix the error/skip EPIC from the benchmarking.

Thanks!

[grst]

I don't think this has anything to do with EPIC, the error message just happens to appear right after the EPIC package gets loaded.

I think I have seen this error message before, but it might have been in the context of a different project. What operating system are you using?

The command above will just run pytest on the internal copy of the immunedeconv project. You could also try to do that manually or run them in rstudio.

[AlmogAngel]

I am running snakemake on a Linux server. This time I tried to run the snakemake --use-conda command but I keep getting errors. I was trying to search google for a solution but nothing worked so far.

(base) [19:48:22] Aran_lab immune_deconvolution_benchmark > snakemake --use-conda -c16
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 16
Rules claiming more threads will be scaled down.
Job stats:
job      count    min threads    max threads
-----  -------  -------------  -------------
book         1              1              1
total        1              1              1

Select jobs to execute...

[Thu May  5 19:48:28 2022]
rule book:
    input: data/ensemble_hgnc.txt, data/immune_cell_reference/immune_cell_reference_tidy.tsv, data/schelker/ascites_bulk_samples.xls, data/schelker/single_cell_schelker.rda, data/schelker/ascites_facs.xlsx, data/racle/GSE93722_RAW/GSM2461007_LAU1255.genes.results.txt, data/racle/GSE93722_RAW/GSM2461009_LAU1314.genes.results.txt, data/racle/GSE93722_RAW/GSM2461005_LAU355.genes.results.txt, data/racle/GSE93722_RAW/GSM2461003_LAU125.genes.results.txt, data/racle/racle2017_flow_cytometry.xlsx, data/hoek/hoek_quantiseq.RData, data/hoek/hoek_star_rsem.rds, data/hoek/HoekPBMC_gtruth.RData, notebooks/36_spillover_marker_genes.Rmd, notebooks/10_methods_validity.Rmd, notebooks/20_validation_study.Rmd, notebooks/05_input_data.Rmd, notebooks/70_pub_figures.Rmd, notebooks/15_mixing_study.Rmd, notebooks/_tcga_comparison.Rmd, notebooks/_main.Rmd, notebooks/30_sensitivity_analysis.Rmd, notebooks/_single_cell_marker_genes_nk.Rmd, notebooks/_pres_plots.Rmd, notebooks/99_bibliography.Rmd, notebooks/index.Rmd, notebooks/_impact_of_preprocessing.Rmd, notebooks/35_spillover_analysis.Rmd, notebooks/_preprocess_single_cell_schelker.Rmd, notebooks/_marker_genes_background.Rmd, notebooks/_mrna_bias.Rmd, notebooks/bibliography.bib, notebooks/_bookdown.yml, notebooks/_output.yml
    output: results/book/index.html, results/cache/.dir, results/figures/schelker_single_cell_tsne.pdf, results/figures/spillover_migration_chart.jpg, results/figures/spillover_migration_all.pdf, results/tables/mixing_study_correlations.tsv, results/tables/spillover_signal_noise.tsv
    jobid: 0
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/e20f3dd15de93fd6e3f455dccfc211bf

processing file: _main.Rmd
  |                                                                 |   0%
   inline R code fragments

  |                                                                 |   1%
label: setup (with options) 
List of 1
 $ include: logi FALSE

terminate called after throwing an instance of 'std::runtime_error'
  what():  Mutex creation failed
[Thu May  5 19:48:38 2022]
Error in rule book:
    jobid: 0
    output: results/book/index.html, results/cache/.dir, results/figures/schelker_single_cell_tsne.pdf, results/figures/spillover_migration_chart.jpg, results/figures/spillover_migration_all.pdf, results/tables/mixing_study_correlations.tsv, results/tables/spillover_signal_noise.tsv
    conda-env: /bigdata/almogangel/immune_deconvolution_benchmark/.snakemake/conda/e20f3dd15de93fd6e3f455dccfc211bf
    shell:

    touch results/cache/.dir
    rm -f results/book/figures && ln -s ../figures results/book/figures
    cd notebooks && Rscript -e "bookdown::render_book('index.Rmd')"

        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job book since they might be corrupted:
results/cache/.dir
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

[grst]

From what I can tell from googling, the issues arises from an incompatible glibc version installed on your system. Theoretically conda should deal with all dependencies, but sometimes it still depends on system libraries.

If you have no chance to upgrade the glibc version, your best bet is to try it on a different system with a different linux distribution. You could also try to start a docker container (e.g. https://hub.docker.com/r/continuumio/miniconda3) and try to execute the pipeline within.

[AlmogAngel]

From what I can tell from googling, the issues arises from an incompatible glibc version installed on your system. Theoretically conda should deal with all dependencies, but sometimes it still depends on system libraries.

If you have no chance to upgrade the glibc version, your best bet is to try it on a different system with a different linux distribution. You could also try to start a docker container (e.g. https://hub.docker.com/r/continuumio/miniconda3) and try to execute the pipeline within.

Thanks again,

I spent a few more hours debugging this, but unfortunately, no luck. After scanning everything on google, seems that you are right, it is something to do with glibc core libraries. However, those posts are all related to old versions of glibc and my system already uses a newer version.

I am wasn't sure what else I can do, so I was trying to run this pipeline locally on my Mac, again no luck :(

Next week I will ask my server admin to install docker and see if it can help. Although, I think that docker also uses core libraries.