Segmentation fault following Matplotlib GUI usage warning

[AnthonyPiot91]

Snakemake runs into a segmentation fault when trying to obtain results files for the macrosynteny plots between two or more genomes.

Specifically, the error occurs during the "plot_synteny_nocolor" rule, after the following warning: UserWarning: Starting a Matplotlib GUI outside of the main thread will likely fail.

I was able to obtain the graph using the code associated with this rule outside snakemake, therefore, it seems to me like a compatibility problem between snakemake and matplotlib or related packages. I think it would be useful to have a list of dependencies in the documentation we are certain are working together for this program. It might help to solve this issues more readily.

I am running the program within a conda environment with the following packages: odp_packages.txt

I tried downgrading snakemake version 8 to 7 without success.

I am running Ubuntu 22.04.3 LTS from WSL2 on Windows 11.

Here is my config file:

ignore_autobreaks: True       # Skip steps to find breaks in synteny blocks
diamond_or_blastp: "diamond"  # "diamond" or "blastp"
plot_LGs: False                # Plot the ALGs based on the installed databases
plot_sp_sp: True              # Plot the synteny between two species, if False just generates .rbh files

species:
  Bb16152:
    proteins: data/Bb16-15-2/Bb16-15-2_dedup.faa  # required field
    chrom:    data/Bb16-15-2/Bb16-15-2.chrom      # required field
    genome:   data/Bb16-15-2/Bb16-15-2.fasta      # required field

  Bb16232:
    proteins: data/Bb16-23-2/Bb16-23-2_dedup.faa
    chrom:    data/Bb16-23-2/Bb16-23-2.chrom
    genome:   data/Bb16-23-2/Bb16-23-2.fasta

And the log including the error occurence, with the associated command line:

snakemake -c 2 -p --rerun-incomplete --snakefile /mnt/c/Users/piota/Software/odp/scripts/odp

Using shell: /usr/bin/bash
Provided cores: 2
Rules claiming more threads will be scaled down.
Job stats:
job                     count
--------------------  -------
all                         1
plot_synteny_nocolor        1
total                       2

Select jobs to execute...

[Thu Jan 18 12:01:08 2024]
rule plot_synteny_nocolor:
    input: odp/step1-rbh/Bb16152_Bb16232_reciprocal_best_hits.D.FET.rbh, odp/step0-chromsize/analyses/Bb16152_Bb16232.chromsize
    output: odp/step2-figures/synteny_nocolor/Bb16152_Bb16232_xy_reciprocal_best_hits.plotted.rbh, odp/step2-figures/synteny_nocolor/Bb16152_Bb16232_xy_synteny.pdf, odp/step2-figures/synteny_nocolor/Bb16152_Bb16232_yx_synteny.pdf
    jobid: 14
    reason: Missing output files: odp/step2-figures/synteny_nocolor/Bb16152_Bb16232_xy_reciprocal_best_hits.plotted.rbh, odp/step2-figures/synteny_nocolor/Bb16152_Bb16232_xy_synteny.pdf, odp/step2-figures/synteny_nocolor/Bb16152_Bb16232_yx_synteny.pdf
    wildcards: analysis=Bb16152_Bb16232
    resources: tmpdir=/tmp, mem_mb=5000, mem_mib=4769

Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 2
Rules claiming more threads will be scaled down.
Provided resources: mem_mb=5000, mem_mib=4769
Select jobs to execute...
All the Bb16232 scaffolds have been identified in a connected component, continuing
All the Bb16152 scaffolds have been identified in a connected component, continuing
/mnt/c/Users/piota/Software/odp/scripts/odp:1229: UserWarning: Starting a Matplotlib GUI outside of the main thread will likely fail.
  # There shouldn't be but we must check anyway.
All the Bb16152 scaffolds have been identified in a connected component, continuing
All the Bb16232 scaffolds have been identified in a connected component, continuing
/mnt/c/Users/piota/Software/odp/scripts/odp:1229: UserWarning: Starting a Matplotlib GUI outside of the main thread will likely fail.
  # There shouldn't be but we must check anyway.
Segmentation fault
Removing output files of failed job plot_synteny_nocolor since they might be corrupted:
odp/step2-figures/synteny_nocolor/Bb16152_Bb16232_xy_reciprocal_best_hits.plotted.rbh, odp/step2-figures/synteny_nocolor/Bb16152_Bb16232_xy_synteny.pdf, odp/step2-figures/synteny_nocolor/Bb16152_Bb16232_yx_synteny.pdf
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2024-01-18T120106.681478.snakemake.log

Increasing the provided memory and cores to snakemake did not change the problem.

Does anybody have an idea what might be going on here? Thanks

[AnthonyPiot91]

As further information, I originally had an error message related to the Qt platform in addition to the matplotlib warning, at the same step:

qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found

I was able to solve this problem by installing a bunch of libraries from this page

sudo apt-get install '^libxcb.*-dev' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev libxkbcommon-dev libxkbcommon-x11-dev

But now I am encountering the segmentation fault.

[conchoecia]

Hi @AnthonyPiot91 - I developed the software on a machine running linux natively. If you are running linux from within Windows, in my experience that leads to odd errors that aren't seen otherwise. I have never encountered something having to do with a GUI, as the code shouldn't be calling fig.show().

If you are getting a segfault that means that the memory was misallocated or your system ran out of memory. Given how python handles memory I am assuming it is the latter. Are you running this on a laptop?

[AnthonyPiot91]

Hi @conchoecia,

Yes I was running the program locally on a laptop (32G 16cores for info).

However, I was able to successfully run the program on a HPC server running linux using the same conda environment, so I also came to the conclusion that WSL is the culprit here.

I do not have the skills to tinker with how python is interacting with the OS so I'm just gonna keep running the program on a Linux machine.

Thank you for your answer.

[conchoecia]

Seems like this was resolved! Closing - please reopen another issue if you have further problems.