Address already in use

[lucacozzuto]

Dear developer, I'm having the following error when executing this tool on a HPC.

  2024-01-08 15:01:03 : INFO  Using high MCC model
  2024-01-08 15:01:03 : INFO  Log file: None
  Process SyncManager-1:
  Traceback (most recent call last):
    File "/usr/local/lib/python3.10/multiprocessing/process.py", line 315, in _bootstrap
      self.run()
    File "/usr/local/lib/python3.10/multiprocessing/process.py", line 108, in run
      self._target(*self._args, **self._kwargs)
    File "/usr/local/lib/python3.10/multiprocessing/managers.py", line 591, in _run_server
      server = cls._Server(registry, address, authkey, serializer)
    File "/usr/local/lib/python3.10/multiprocessing/managers.py", line 156, in __init__
      self.listener = Listener(address=address, backlog=16)
    File "/usr/local/lib/python3.10/multiprocessing/connection.py", line 453, in __init__
      self._listener = SocketListener(address, family, backlog)
    File "/usr/local/lib/python3.10/multiprocessing/connection.py", line 596, in __init__
      self._socket.bind(address)
  OSError: [Errno 98] Address already in use
  Traceback (most recent call last):
    File "/usr/local/bin/ribodetector_cpu", line 10, in <module>
      sys.exit(main())
    File "/usr/local/lib/python3.10/site-packages/ribodetector/detect_cpu.py", line 822, in main
      seq_pred.detect()
    File "/usr/local/lib/python3.10/site-packages/ribodetector/detect_cpu.py", line 597, in detect
      self.run()
    File "/usr/local/lib/python3.10/site-packages/ribodetector/detect_cpu.py", line 103, in run
      manager = mp.Manager()
    File "/usr/local/lib/python3.10/multiprocessing/context.py", line 57, in Manager
      m.start()
    File "/usr/local/lib/python3.10/multiprocessing/managers.py", line 566, in start
      self._address = reader.recv()
    File "/usr/local/lib/python3.10/multiprocessing/connection.py", line 255, in recv
      buf = self._recv_bytes()
    File "/usr/local/lib/python3.10/multiprocessing/connection.py", line 419, in _recv_bytes
      buf = self._recv(4)
    File "/usr/local/lib/python3.10/multiprocessing/connection.py", line 388, in _recv
      raise EOFError
  EOFError

I tried also to reduce the number of jobs to 1 to avoid parallelization but not luck. Can you help me with this, please?

Luca

[dawnmy]

After researching online, I found that the issue stems from Python versions 3.9 and 3.10 generating less random default socket addresses for multiprocessing.Manager() on Unix machines. https://stackoverflow.com/questions/74058016/how-do-you-run-multiple-aws-batch-jobs-with-multiprocessing-mp-manager-witho. As a workaround, you could try downgrading to Python 3.8 using conda in your environment.

[lucacozzuto]

Thank you @dawnmy ! I was using that version of python because it is the one that is embedded in biocontainers that is automatically generated from bioconda (https://github.com/bioconda/bioconda-recipes/blob/11dfefa5468e9425df9b3aed5c4c4765f57bb689/recipes/ribodetector/meta.yaml) I'll try to point them to this issue. Best,

Luca

[lucacozzuto]

I made a pull request: https://github.com/bioconda/bioconda-recipes/pull/45133

[dawnmy]

I will fix this issue in the next release with a workaround mentioned here:

https://github.com/python/cpython/issues/97514

by adding this line

multiprocessing.util.abstract_sockets_supported = False

For now, you can install ribodetector using pip in a env with Python3.8

[dawnmy]

now it should be fixed with version 0.3.1. This version is already available in pip but may need few days to be available in coonda. Let me know if this new version works.