Error in 'prepare-rpbp-genome'

[PSSUN]

When I try to run the example data in first step:

prepare-rpbp-genome c-elegans-test.yaml --overwrite

I met this error:

subprocess.CalledProcessError: Command 'split-bed12-blocks /home/sun/sps/c-elegans-chrI-example/WBcel235.79.chrI/WBcel235.79.chrI.annotated.bed.gz /home/sun/sps/c-elegans-chrI-example/WBcel235.79.chrI/transcript-index/WBcel235.79.chrI.exons.annotated.bed.gz --num-cpus 1 --logging-level WARNING --file-logging-level NOTSET --stdout-logging-level NOTSET --stderr-logging-level NOTSET' returned non-zero exit status 1.

Then I try to run this step independently: split-bed12-blocks /home/sun/sps/c-elegans-chrI-example/WBcel235.79.chrI/WBcel235.79.chrI.annotated.bed.gz /home/sun/sps/c-elegans-chrI-example/WBcel235.79.chrI/transcript-index/WBcel235.79.chrI.exons.annotated.bed.gz --num-cpus 1 --logging-level WARNING --file-logging-level NOTSET --stdout-logging-level NOTSET --stderr-logging-level NOTSET The error is : TSET --stdout-logging-level NOTSET --stderr-logging-level NOTSET Traceback (most recent call last): File "/usr/local/bin/split-bed12-blocks", line 11, in ............. File "/usr/lib/python3.6/gzip.py", line 463, in read if not self._read_gzip_header(): File "/usr/lib/python3.6/gzip.py", line 411, in _read_gzip_header raise OSError('Not a gzipped file (%r)' % magic) OSError: Not a gzipped file (b'ic')

It seems that file ''WBcel235.79.chrI.annotated.bed.gz" is a wrong gzipped file, it can't be extract.

[PSSUN]

I have tried to extract WBcel235.79.chrI.annotated.bed.gz on Mac and Windows, it can't be open, I guess there is something wrong in the step which make this file.

[eboileau]

Dear @Sunpeisen, I cannot reproduce this error. You might have a corrupted file, in which case deleting it and retrying might help. You can use directly

gtf-to-bed12 WBcel235.79.chrI.gtf WBcel235.79.chrI.bed.gz

and then verify the output. This file is written as a pandas dataframe using the gzip module, so nothing special. On linux, you can use file WBcel235.79.chrI.annotated.bed.gz to see if it is indeed gzip compressed data or not.

[PSSUN]

Dear @eboileau :
I want to uninstall rpbp and install again, but after I used: pip3 uninstall misc bio-utils riboutils rpbp It told me :
Successfully uninstalled rpbp-1.1.12 but I input'prep'and use TAB , I still can find 'prepare-rpbp-genome' in list. How can I REMOVE THEM CLEARLY. And install rpbp again.

[eboileau]

Could it be that you also have another prior installation? You could also try deleting these manually. If the packages were installed in a dedicated virtual environment, this environment can simply be removed, and I strongly advise to always use dedicated virtual environments, see Virtual environment installation.

[eboileau]

Hi, have you re-installed successfully? Please let me know, so that we can update the status of this issue.

[PSSUN]

Sorry for replay so late. I have uninstalled rp-bp and re-install successfully. But this error appeared again, however I decide to change another computer to try again. Thank you for your answer.

[PSSUN]

Dear Author,

I have reinstalled rp-bp and run 'prepare-rpbp-genome' step successfully. When I try to run the step 'run-all-rpbp-instances', the command line is: run-all-rpbp-instances c-elegans-test.yaml --overwrite --num-cpus 6 --logging-level INFO --merge-replicates --run-replicates --keep-intermediate-files

And the error in this step is :

... INFO rpbp.translation_prediction.estimate_orf_bayes_factors 2018-07-24 15:17:31,603 : Reading and filtering ORFs INFO rpbp.translation_prediction.estimate_orf_bayes_factors 2018-07-24 15:17:31,892 : Number of regions after filtering: 438 joblib.externals.loky.process_executor._RemoteTraceback: """ Traceback (most recent call last): File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/externals/loky/process_executor.py", line 402, in _process_worker r = call_item.fn(*call_item.args, *call_item.kwargs) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 561, in call return self.func(args, **kwargs) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/parallel.py", line 224, in call for func, args, kwargs in self.items] File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/parallel.py", line 224, in for func, args, kwargs in self.items] File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/rpbp/translation_prediction/estimate_orf_bayes_factors.py", line 292, in get_all_bayes_factors_args shape=profiles_shape, copy=False) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/scipy/sparse/compressed.py", line 61, in init maxval = max(shape) TypeError: 'int' object is not iterable ...

I don't know how to solve this problem.

[eboileau]

Hi, Could you please include the complete output, and also what OS you are using, list packages in your virtual environment (pip freeze or pip list). Have you looked at the output of the installation log, to make sure all went well?

[PSSUN]

OS: ubuntu 18.04

complete output:

/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, *kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(args, kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, kwds) 30430 reads; of these: 30430 (100.00%) were unpaired; of these: 20523 (67.44%) aligned 0 times 9907 (32.56%) aligned exactly 1 time 0 (0.00%) aligned >1 times 32.56% overall alignment rate Jul 25 09:34:44 ..... started STAR run Jul 25 09:34:44 ..... loading genome Jul 25 09:34:45 ..... processing annotations GTF Jul 25 09:34:45 ..... inserting junctions into the genome indices Jul 25 09:34:54 ..... started mapping Jul 25 09:34:56 ..... started sorting BAM Jul 25 09:34:56 ..... finished successfully WARNING root 2018-07-25 09:34:56,840 : [utils.create_symlink]: file already exists at: '/home/sun/sps/01_software/rp-bp/c-elegans-example/without-rrna-mapping/c-elegans-rep-1.test.bam'. It will be removed /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, *kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(args, kwds) 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10163/10163 [00:00<00:00, 531929.61it/s] /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, *kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(args, kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, *kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(args, kwds) 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10163/10163 [00:00<00:00, 604857.27it/s] 0%| | 0/100 [00:00<?, ?it/s]/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, *kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(args, kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, kwds) 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100/100 [00:06<00:00, 15.04it/s] 53407it [00:11, 4699.51it/s] /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, *kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(args, kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, kwds) /usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, *kwds) joblib.externals.loky.process_executor._RemoteTraceback: """ Traceback (most recent call last): File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/externals/loky/process_executor.py", line 402, in _process_worker r = call_item.fn(call_item.args, call_item.kwargs) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 561, in call return self.func(*args, **kwargs) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/parallel.py", line 224, in call for func, args, kwargs in self.items] File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/parallel.py", line 224, in for func, args, kwargs in self.items] File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/rpbp/translation_prediction/estimate_orf_bayes_factors.py", line 292, in get_all_bayes_factors_args shape=profiles_shape, copy=False) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/scipy/sparse/compressed.py", line 61, in init maxval = max(shape) TypeError: 'int' object is not iterable """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/sun/sps/01_software/vene/bin/estimate-orf-bayes-factors", line 11, in load_entry_point('rpbp==1.1.12', 'console_scripts', 'estimate-orf-bayes-factors')() File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/rpbp/translation_prediction/estimate_orf_bayes_factors.py", line 456, in main progress_bar=True File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/misc/parallel.py", line 179, in apply_parallel_split res = apply_parallel_groups(split_groups, num_procs, func, *args, progress_bar=progress_bar) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/misc/parallel.py", line 101, in apply_parallel_groups for name,group in tqdm.tqdm(groups, total=len(groups), leave=True, file=sys.stdout)) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/parallel.py", line 962, in call self.retrieve() File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/parallel.py", line 865, in retrieve self._output.extend(job.get(timeout=self.timeout)) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 515, in wrap_future_result return future.result(timeout=timeout) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/externals/loky/_base.py", line 431, in result return self.__get_result() File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/joblib/externals/loky/_base.py", line 382, in get_result raise self._exception TypeError: 'int' object is not iterable Exception ignored in: <bound method tqdm.del of 4%|▍ | 4/100 [00:00<00:10, 9.41it/s]> Traceback (most recent call last): File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/tqdm/_tqdm.py", line 882, in del__ File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/tqdm/_tqdm.py", line 1087, in close File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/tqdm/_tqdm.py", line 439, in _decr_instances File "/usr/lib/python3.6/_weakrefset.py", line 109, in remove KeyError: <weakref at 0x7fceb727b9f8; to 'tqdm' at 0x7fceb74c4208> Traceback (most recent call last): File "/home/sun/sps/01_software/vene/bin/predict-translated-orfs", line 11, in load_entry_point('rpbp==1.1.12', 'console_scripts', 'predict-translated-orfs')() File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/rpbp/translation_prediction/predict_translated_orfs.py", line 255, in main file_checkers=file_checkers, overwrite=args.overwrite, call=call) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/misc/shell_utils.py", line 345, in call_if_not_exists ret_code = check_call(cmd, call=call, raise_on_error=raise_on_error) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/misc/shell_utils.py", line 193, in check_call return check_call_step(cmd, call=call, raise_on_error=raise_on_error) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/misc/shell_utils.py", line 176, in check_call_step raise subprocess.CalledProcessError(ret_code, cmd) subprocess.CalledProcessError: Command 'estimate-orf-bayes-factors /home/sun/sps/01_software/rp-bp/c-elegans-example/orf-profiles/c-elegans-rep-1.test-unique.length-29.offset-12.profiles.mtx.gz /home/sun/sps/01_software/rp-bp/c-elegans-chrI-example/WBcel235.79.chrI/transcript-index/WBcel235.79.chrI.genomic-orfs.atg-only.bed.gz /home/sun/sps/01_software/rp-bp/c-elegans-example/orf-predictions/c-elegans-rep-1.test-unique.length-29.offset-12.frac-0.2.rw-0.bayes-factors.bed.gz --translated-models /home/sun/.local/share/rpbp/rpbp_models/translated/periodic-gaussian-mixture.pkl --untranslated-models /home/sun/.local/share/rpbp/rpbp_models/untranslated/gaussian-naive-bayes.pkl --logging-level WARNING --file-logging-level NOTSET --stdout-logging-level NOTSET --stderr-logging-level NOTSET --min-length 0 --max-length 0 --min-profile 5 --fraction 0.2 --reweighting-iterations 0 --seed 8675309 --iterations 200 --chains 2 --num-cpus 2' returned non-zero exit status 1. Traceback (most recent call last): File "/home/sun/sps/01_software/vene/bin/run-rpbp-pipeline", line 11, in load_entry_point('rpbp==1.1.12', 'console_scripts', 'run-rpbp-pipeline')() File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/rpbp/run_rpbp_pipeline.py", line 148, in main shell_utils.check_call(cmd) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/misc/shell_utils.py", line 193, in check_call return check_call_step(cmd, call=call, raise_on_error=raise_on_error) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/misc/shell_utils.py", line 176, in check_call_step raise subprocess.CalledProcessError(ret_code, cmd) subprocess.CalledProcessError: Command 'predict-translated-orfs c-elegans-test.yaml c-elegans-rep-1 --num-cpus 2 --overwrite --logging-level WARNING --file-logging-level NOTSET --stdout-logging-level NOTSET --stderr-logging-level NOTSET' returned non-zero exit status 1. Traceback (most recent call last): File "/home/sun/sps/01_software/vene/bin/run-all-rpbp-instances", line 11, in load_entry_point('rpbp==1.1.12', 'console_scripts', 'run-all-rpbp-instances')() File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/rpbp/run_all_rpbp_instances.py", line 181, in main job_id = slurm.check_sbatch(cmd, args=args) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/misc/slurm.py", line 188, in check_sbatch shell_utils.check_call(cmd, call=call) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/misc/shell_utils.py", line 193, in check_call return check_call_step(cmd, call=call, raise_on_error=raise_on_error) File "/home/sun/sps/01_software/vene/lib/python3.6/site-packages/misc/shell_utils.py", line 176, in check_call_step raise subprocess.CalledProcessError(ret_code, cmd) subprocess.CalledProcessError: Command 'run-rpbp-pipeline /home/sun/sps/01_software/rp-bp/c-elegans-chrI-example/c-elegans.test-chrI.rep-1.fastq.gz c-elegans-test.yaml c-elegans-rep-1 --num-cpus 2 --overwrite --logging-level WARNING --file-logging-level NOTSET --stdout-logging-level NOTSET --stderr-logging-level NOTSET --star-executable STAR --star-read-files-command zcat --keep-intermediate-files --mem 2G' returned non-zero exit status 1.

pip list:

Package Version

---

appdirs 1.4.3
asn1crypto 0.24.0
bcrypt 3.1.4
bio-utils 0.2.6
biopython 1.72
certifi 2018.4.16 cffi 1.11.5
chardet 3.0.4
click 6.7
cloudpickle 0.5.3
cryptography 2.3
cycler 0.10.0
Cython 0.28.4
dask 0.18.2
decorator 4.3.0
distributed 1.22.0
docopt 0.6.2
et-xmlfile 1.0.1
fastparquet 0.1.5
graphviz 0.8.4
HeapDict 1.0.0
idna 2.7
jdcal 1.4
joblib 0.12.1
kiwisolver 1.0.1
llvmlite 0.24.0
locket 0.2.0
matplotlib 2.2.2
matplotlib-venn 0.11.5
misc 0.2.8
more-itertools 4.2.0
msgpack 0.5.6
mygene 3.0.0
networkx 2.1
nltk 3.3
numba 0.39.0
numexpr 2.6.6
numpy 1.15.0
openpyxl 2.5.4
pandas 0.23.3
paramiko 2.4.1
partd 0.3.8
patsy 0.5.0
pip 18.0
pkg-resources 0.0.0
psutil 5.4.6
pyasn1 0.4.3
pycparser 2.18
pydot 1.2.4
pyfasta 0.5.2
PyNaCl 1.2.1
pyparsing 2.2.0
pysam 0.14.1
pystan 2.16.0.0 python-dateutil 2.7.3
pytz 2018.5
PyYAML 3.13
requests 2.19.1
riboutils 0.2.7
rpbp 1.1.12
scikit-learn 0.19.2
scipy 1.1.0
seaborn 0.9.0
setuptools 39.0.1
six 1.11.0
sklearn 0.0
sortedcontainers 2.0.4
spur 0.3.20
statsmodels 0.9.0
tables 3.4.4
tblib 1.3.2
thrift 0.11.0
toolz 0.9.0
tornado 5.1
tqdm 4.23.4
urllib3 1.23
xlrd 1.1.0
zict 0.1.3

installation output:

Successfully installed appdirs-1.4.3 asn1crypto-0.24.0 bcrypt-3.1.4 bio-utils-0.2.6 biopython-1.72 certifi-2018.4.16 cffi-1.11.5 chardet-3.0.4 click-6.7 cloudpickle-0.5.3 cryptography-2.3 cycler-0.10.0 cython-0.28.4 dask-0.18.2 decorator-4.3.0 distributed-1.22.0 docopt-0.6.2 et-xmlfile-1.0.1 fastparquet-0.1.5 graphviz-0.8.4 heapdict-1.0.0 idna-2.7 jdcal-1.4 joblib-0.12.1 kiwisolver-1.0.1 llvmlite-0.24.0 locket-0.2.0 matplotlib-2.2.2 matplotlib-venn-0.11.5 misc-0.2.8 more-itertools-4.2.0 msgpack-0.5.6 mygene-3.0.0 networkx-2.1 nltk-3.3 numba-0.39.0 numexpr-2.6.6 numpy-1.15.0 openpyxl-2.5.4 pandas-0.23.3 paramiko-2.4.1 partd-0.3.8 patsy-0.5.0 psutil-5.4.6 pyasn1-0.4.3 pycparser-2.18 pydot-1.2.4 pyfasta-0.5.2 pynacl-1.2.1 pyparsing-2.2.0 pysam-0.14.1 pystan-2.16.0.0 python-dateutil-2.7.3 pytz-2018.5 pyyaml-3.13 requests-2.19.1 riboutils-0.2.7 rpbp-1.1.12 scikit-learn-0.19.2 scipy-1.1.0 seaborn-0.9.0 six-1.11.0 sklearn-0.0 sortedcontainers-2.0.4 spur-0.3.20 statsmodels-0.9.0 tables-3.4.4 tblib-1.3.2 thrift-0.11.0 toolz-0.9.0 tornado-5.1 tqdm-4.23.4 urllib3-1.23 xlrd-1.1.0 zict-0.1.3 Cleaning up... Removed build tracker '/tmp/pip-req-tracker-dd66il8h'

[PSSUN]

The warning

/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
return f(*args, **kwds)

also appeared in step 'prepare-rpbp-genome' I have changed the version of numpy to 1.14.5 , this warning disappered. But the error still exist

[eboileau]

Hi,

The warnings

RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88

could be due to version conflict in some of the installed packages, depending on how Rp-Bp was installed (locally, python virtual environment or conda, you have not specified). I have not had these before. They are likely due to importing scipy (or another package) that was compiled against an older numpy than is installed, see e.g. Cython warnings. I think they could be safely ignored for the moment, but in a next release of Rp-Bp, we will fix versions for some of the critical required packages.

It looks like the shape of the CSR profile matrix is not a 2-tuple but an integer, which then causes the TypeError. I honestly don't know why this would happen, since shape is a 2-tuple, and even though the profiles.mtx.gz would be empty, scipy would return shape=(0, 0). The explanation could be that profiles_shape falls back to the default integer value in get_all_bayes_factors_args, because this function cannot see the ctypes array allocated from shared memory...!?! The problem, though, is that I cannot reproduce this error at all.

You still have not mentioned which OS you are using?

It might be worth trying to re-install (Apologies for inconveniences. Note that this could easily be done via virtual environments), but first you would need to modify the setup.py files for misc, riboutils, bio_utils and rpbp to fix versions of external_requirements to some of the packages:

biopython==1.70
Cython==0.28
dask==0.17.1
distributed==1.21.3
fastparquet==0.1.4
joblib==0.11
llvmlite==0.22.0
numpy==1.14.2
pandas==0.22.0
patsy==0.5.0
pysam==0.14
scikit-learn==0.19.1
scipy==1.0.0
tornado==5.0
tqdm==4.19.7
sklearn==0.0
statsmodels==0.8.0
psutil==5.4.3 

[bmmalone]

For reference, I've recently had the numpy.dtype size changed warning in other projects (have not checked on rpbp). Those still behaved okay. I'm not sure if rpbp may be different, but just to pass that along.

[PSSUN]

Dear author, When I uesd 6 cpus to run rp-bp, command line: run-all-rpbp-instances c-elegans-test.yaml --overwrite --num-cpus 6 --logging-level INFO --merge-replicates --run-replicates --keep-intermediate-files The error exist, but when I used only 1 cpu to run rp-bp, command line: run-all-rpbp-instances c-elegans-test.yaml --overwrite --logging-level INFO --merge-replicates --run-replicates --keep-intermediate-files Rp-Bp can run successfully, if there something wrong with multprocess of rp-bp?

I used Linux operating system, the version is ubuntu 18.04, cpu is intel i5-8800, 6 core and 6 thread.

[eboileau]

Since you experience this problem only in get_all_bayes_factors_args, as I said it would appear that globals are not made available to all worker processes. Even assuming that this would effectively be the cause of the error, it's hard to track down, since it is not reproducible.

Have you tried fixing versions of the packages in a fresh virtual environment? This would be the first step. One hint: joblib has changed it's default backend from multiprocessing to loky since version 0.12.

Also, just to clarify, multiprocessing is not a component of Rp-Bp, but part of the Python Standard Library.

In all cases, this has diverged from the initial topic, so I will close this issue. I will open another issue regarding versions requirements to be fixed. If you still experience errors after fixing the packages, please open a new issue.