search

macs3-project / MACS

MACS -- Model-based Analysis of ChIP-Seq
https://macs3-project.github.io/MACS/
BSD 3-Clause "New" or "Revised" License
708 stars 268 forks source link

Bug: Runtime error pybind11::handle::inc_ref() PyGILState_Check() failure. #666

Open Kiliankleemann opened 1 week ago

Kiliankleemann commented 1 week ago

Error message below (bottom): INFO @ 21 Oct 2024 16:38:48: [107 MB]

Command line: hmmratac -i BED_files_sorted/9.bedpe -f BEDPE -n macs3_output/9

Random seed selected as: 10151

Use --hmm-type to select a Gaussian ('gaussian') or Poisson ('poisson') model for the hidden markov model in HMMRATAC. Default: 'gaussian'.

INFO @ 21 Oct 2024 16:38:48: [107 MB] #1 Read fragments from BEDPE file... INFO @ 21 Oct 2024 16:38:49: [197 MB] 1000000 fragments parsed INFO @ 21 Oct 2024 16:38:50: [213 MB] 2000000 fragments parsed INFO @ 21 Oct 2024 16:38:51: [222 MB] 3000000 fragments parsed INFO @ 21 Oct 2024 16:38:52: [226 MB] 4000000 fragments parsed INFO @ 21 Oct 2024 16:38:54: [226 MB] 5000000 fragments parsed INFO @ 21 Oct 2024 16:38:55: [227 MB] 6000000 fragments parsed INFO @ 21 Oct 2024 16:38:56: [228 MB] 7000000 fragments parsed INFO @ 21 Oct 2024 16:38:57: [229 MB] 8000000 fragments parsed INFO @ 21 Oct 2024 16:38:58: [239 MB] 9000000 fragments parsed INFO @ 21 Oct 2024 16:39:00: [245 MB] 10000000 fragments parsed INFO @ 21 Oct 2024 16:39:01: [259 MB] 11000000 fragments parsed INFO @ 21 Oct 2024 16:39:02: [263 MB] 12000000 fragments parsed INFO @ 21 Oct 2024 16:39:03: [263 MB] 13000000 fragments parsed INFO @ 21 Oct 2024 16:39:04: [273 MB] 14000000 fragments parsed INFO @ 21 Oct 2024 16:39:06: [273 MB] 15000000 fragments parsed INFO @ 21 Oct 2024 16:39:07: [279 MB] 16000000 fragments parsed INFO @ 21 Oct 2024 16:39:08: [279 MB] 17000000 fragments parsed INFO @ 21 Oct 2024 16:39:09: [289 MB] 18000000 fragments parsed INFO @ 21 Oct 2024 16:39:10: [305 MB] 19000000 fragments parsed INFO @ 21 Oct 2024 16:39:24: [358 MB] #2 Use EM algorithm to estimate means and stddevs of fragment lengths INFO @ 21 Oct 2024 16:39:24: [358 MB] # for mono-, di-, and tri-nucleosomal signals... INFO @ 21 Oct 2024 16:39:24: [359 MB] # A random seed 10151 has been used in the sampling function INFO @ 21 Oct 2024 16:39:26: [428 MB] # Downsampled 1904083 fragments will be used for EM training... INFO @ 21 Oct 2024 16:45:35: [428 MB] # Reached convergence after 14 iterations INFO @ 21 Oct 2024 16:45:35: [428 MB] # The means and stddevs after EM: INFO @ 21 Oct 2024 16:45:35: [428 MB] # short mono di tri INFO @ 21 Oct 2024 16:45:35: [428 MB] # means: 50 210.1 382.4 611.7 INFO @ 21 Oct 2024 16:45:35: [428 MB] # stddevs: 20 45.8 45.7 101.8 INFO @ 21 Oct 2024 16:45:35: [428 MB] # Compute the weights for each fragment length for each of the four signal types INFO @ 21 Oct 2024 16:45:42: [428 MB] # Generate short, mono-, di-, and tri-nucleosomal signals INFO @ 21 Oct 2024 16:56:22: [928 MB] # Pile up all fragments INFO @ 21 Oct 2024 16:56:43: [928 MB] # Convert pileup to fold-change over average signal INFO @ 21 Oct 2024 16:56:49: [928 MB] #3 Look for training set from 19146841 fragments INFO @ 21 Oct 2024 16:56:49: [928 MB] # Call peak above within fold-change range of 10 and 20. INFO @ 21 Oct 2024 16:56:49: [928 MB] # The minimum length of the region is set as the average template/fragment length in the dataset: 592 INFO @ 21 Oct 2024 16:56:49: [928 MB] # The maximum gap to merge nearby significant regions is set as the flanking size to extend training regions: 1000 INFO @ 21 Oct 2024 16:56:53: [928 MB] # Total training regions called after applying the lower cutoff 10: 12930 INFO @ 21 Oct 2024 16:56:53: [928 MB] # Total training regions after filtering with upper cutoff 20: 3820 INFO @ 21 Oct 2024 16:56:53: [928 MB] # IMPORTANT INFO @ 21 Oct 2024 16:56:53: [928 MB] # Please review the cutoff analysis result in macs3_output/9_cutoff_analysis.tsv to verify INFO @ 21 Oct 2024 16:56:53: [928 MB] # if the choices of lower, upper and prescanning cutoff are appropriate. INFO @ 21 Oct 2024 16:56:53: [928 MB] # Please read the message in the section 'Choices of cutoff values' by running INFO @ 21 Oct 2024 16:56:53: [928 MB] # macs3 hmmratac -h for detail. INFO @ 21 Oct 2024 16:56:53: [928 MB] # **** INFO @ 21 Oct 2024 16:57:39: [928 MB] # We randomly pick 1000 regions for training INFO @ 21 Oct 2024 16:57:39: [928 MB] # We expand the training regions with 1000 basepairs and merge overlap INFO @ 21 Oct 2024 16:57:39: [928 MB] #4 Train Hidden Markov Model with Multivariate Gaussian Emission INFO @ 21 Oct 2024 16:57:39: [928 MB] # Extract signals in training regions with bin size of 10 INFO @ 21 Oct 2024 16:57:50: [1116 MB] # Use Baum-Welch algorithm to train the HMM pybind11::handle::inc_ref() is being called while the GIL is either not held or invalid. Please see https://pybind11.readthedocs.io/en/stable/advanced/misc.html#common-sources-of-global-interpreter-lock-errors for debugging advice. If you are convinced there is no bug in your code, you can #define PYBIND11_NO_ASSERT_GIL_HELD_INCREF_DECREF to disable this check. In that case you have to ensure this #define is consistently used for all translation units linked into a given pybind11 extension, otherwise there will be ODR violations. The failing pybind11::handle::inc_ref() call was triggered on a numpy.ndarray object. Traceback (most recent call last): File "/Users/kiliankleemann/anaconda3/envs/macs3/bin/macs3", line 983, in main() File "/Users/kiliankleemann/anaconda3/envs/macs3/bin/macs3", line 97, in main run( args ) File "/Users/kiliankleemann/anaconda3/envs/macs3/lib/python3.12/site-packages/MACS3/Commands/hmmratac_cmd.py", line 305, in run hmm_model = hmm_training( training_data, training_data_lengths, random_seed = options.hmm_randomSeed, hmm_type = options.hmm_type ) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "MACS3/Signal/HMMR_HMM.pyx", line 60, in MACS3.Signal.HMMR_HMM.hmm_training File "MACS3/Signal/HMMR_HMM.pyx", line 70, in MACS3.Signal.HMMR_HMM.hmm_training File "/Users/kiliankleemann/anaconda3/envs/macs3/lib/python3.12/site-packages/hmmlearn/base.py", line 485, in fit stats, curr_logprob = self._do_estep(X, lengths) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kiliankleemann/anaconda3/envs/macs3/lib/python3.12/site-packages/hmmlearn/base.py", line 764, in _do_estep lattice, logprob, posteriors, fwdlattice, bwdlattice = impl(sub_X) ^^^^^^^^^^^ File "/Users/kiliankleemann/anaconda3/envs/macs3/lib/python3.12/site-packages/hmmlearn/base.py", line 884, in _fit_log log_prob, fwdlattice = _hmmc.forward_log( ^^^^^^^^^^^^^^^^^^ RuntimeError: pybind11::handle::inc_ref() PyGILState_Check() failure.

System (please complete the following information):

taoliu commented 1 week ago

@Kiliankleemann It seems to be caused by dependencies of numpy which has a major version update to version 2. Could you try to upgrade macs3 to 3.0.2 and try again? You need to use pip (https://pypi.org/project/MACS3/) to upgrade it. Let me know if the problem still exists. The reason may be that in 3.0.1 version, we only set the version of numpy to be >= 1.24, and the conda system may install the newest numpy 2 which seems to not work well with another dependency -- hmmlearn.