search

Teichlab / cellphonedb

MIT License
338 stars 105 forks source link

Broken Pipe Errror #302

Closed YinCY123 closed 3 years ago

YinCY123 commented 3 years ago

Hello did anybody meet this problem? recently, I'm running cellphonedb on different three data, two of them are fine, but one of them repeatedly with the Broken Pipe error, does anybody come with this error?

The message returns:

[ ][APP][26/04/21-00:08:07][WARNING] Latest local available version is v2.0.0, using it [ ][APP][26/04/21-00:08:07][WARNING] User selected downloaded database v2.0.0 is available, using it [ ][CORE][26/04/21-00:08:07][INFO] Initializing SqlAlchemy CellPhoneDB Core [ ][CORE][26/04/21-00:08:07][INFO] Using custom database at /home/yincy/.cpdb/releases/v2.0.0/cellphone.db [ ][APP][26/04/21-00:08:07][INFO] Launching Method cpdb_statistical_analysis_local_method_launcher [ ][APP][26/04/21-00:08:07][INFO] Launching Method _set_paths [ ][APP][26/04/21-00:08:07][INFO] Launching Method _load_meta_counts [ ][CORE][26/04/21-00:11:17][INFO] Launching Method cpdb_statistical_analysis_launcher [ ][CORE][26/04/21-00:11:17][INFO] Launching Method _counts_validations [ ][CORE][26/04/21-00:11:29][INFO] [Cluster Statistical Analysis] Threshold:0.1 Iterations:1000 Debug-seed:-1 Threads:8 Precision:3 [ ][CORE][26/04/21-00:11:52][INFO] Running Real Analysis [ ][CORE][26/04/21-00:19:35][INFO] Running Statistical Analysis Killed (biopy) yincy@yincy:~$ Process ForkPoolWorker-3: Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 131, in worker put((job, i, result)) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 397, in _send_bytes self._send(buf) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 136, in worker put((job, i, (False, wrapped))) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 404, in _send_bytes self._send(header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe Process ForkPoolWorker-2: Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 131, in worker put((job, i, result)) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 395, in _send_bytes self._send(pre_header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 136, in worker put((job, i, (False, wrapped))) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 404, in _send_bytes self._send(header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe Process ForkPoolWorker-4: Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 131, in worker put((job, i, result)) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 395, in _send_bytes self._send(pre_header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 136, in worker put((job, i, (False, wrapped))) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 404, in _send_bytes self._send(header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe Process ForkPoolWorker-8: Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 131, in worker put((job, i, result)) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 395, in _send_bytes self._send(pre_header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 136, in worker put((job, i, (False, wrapped))) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 404, in _send_bytes self._send(header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe Process ForkPoolWorker-6: Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 131, in worker put((job, i, result)) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 395, in _send_bytes self._send(pre_header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 136, in worker put((job, i, (False, wrapped))) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 404, in _send_bytes self._send(header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe Process ForkPoolWorker-5: Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 131, in worker put((job, i, result)) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 395, in _send_bytes self._send(pre_header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 136, in worker put((job, i, (False, wrapped))) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 404, in _send_bytes self._send(header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe Process ForkPoolWorker-1: Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 131, in worker put((job, i, result)) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 395, in _send_bytes self._send(pre_header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 136, in worker put((job, i, (False, wrapped))) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 404, in _send_bytes self._send(header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe Process ForkPoolWorker-7: Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 131, in worker put((job, i, result)) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 395, in _send_bytes self._send(pre_header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/pool.py", line 136, in worker put((job, i, (False, wrapped))) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/queues.py", line 368, in put self._writer.send_bytes(obj) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 404, in _send_bytes self._send(header) File "/home/yincy/miniconda3/envs/biopy/lib/python3.8/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe

prete commented 3 years ago

Hi @YinCY123 "Broken pipe" usually means some kind of IOError error, perhaps it got killed and that's why there's no process reading from the other end of a pipe.

Are you running this as a job submitted to a cluster? The first part of the traceback shows:

[ ][CORE][26/04/21-00:19:35][INFO] Running Statistical Analysis
Killed

I think the process got killed and that's what's triggering the broken pip error. Could it be that CellPhoneDB is trying to use more memory than you requested for your job or what's available on your system? Can you try running the same thing again but requesting more memory?

YinCY123 commented 3 years ago

Hi @YinCY123 "Broken pipe" usually means some kind of IOError error, perhaps it got killed and that's why there's no process reading from the other end of a pipe.

Are you running this as a job submitted to a cluster? The first part of the traceback shows:

[ ][CORE][26/04/21-00:19:35][INFO] Running Statistical Analysis
Killed

I think the process got killed and that's what's triggering the broken pip error. Could it be that CellPhoneDB is trying to use more memory than you requested for your job or what's available on your system? Can you try running the same thing again but requesting more memory?

Thanks @prete the problem was solved, as you said, this problem caused by the limited memory. Thanks