Open laytonjbgmail opened 2 years ago
classner
commented
2 years ago Hi @laytonjbgmail ,
That's interesting! I assume you are importing and using PyMP already in a multi-process environment in this failure case? In PyMP, I do not configure a port and I am not sure what port is used by default by the multiprocessing package.
Best, Christoph
laytonjbgmail
commented
2 years ago Christoph,
Yes, I'm using pymp in an earlier part of the code. It works correctly and I can see the various processes being created, running, and then stopping. I'm trying to create a re-producer since the full code is very large as is the data set.
I wonder if there is a way to specify the port for the multiprocessor package? I'm definitely not the best coder, but I'll look around a bit.
Thanks!
Jeff
classner
commented
9 months ago Hi Jeff!
Getting back on this stale issue: do you have a reproducer, or can I close this issue in the next week assuming it is resolved in the meantime?
Best, Christoph
Good morning,
I get an error when running a code using pymp. The error is:
Traceback (most recent call last): File "test8_chunk.py", line 3478, in
currentfigure2, currenttable2, currenttable_ascii2 = iops_output_stats(f_iops_stats, CmdCounter, BeginTime, EndTime, WRITE, READ, TOTAL,\
File "/raid/jlayton/TEMP5/iops_summary_output.py", line 125, in iops_output_stats
perf_summary, thread_io_func2 = iops_getstats(CmdCounter, BeginTime, EndTime, WRITE, READ, TOTAL, VFLAGS, MinIOPS, \
File "/raid/jlayton/TEMP5/iops_summary_output.py", line 1955, in iops_getstats
with pymp.Parallel(nchunk_leftover) as p:
File "/usr/local/lib/python3.8/dist-packages/pymp/init.py", line 56, in init
self._dynamic_queue = _shared.queue()
File "/usr/local/lib/python3.8/dist-packages/pymp/shared.py", line 58, in queue
return _MANAGER.Queue(*args, *kwargs)
File "/usr/lib/python3.8/multiprocessing/managers.py", line 740, in temp
token, exp = self._create(typeid, args, **kwds)
File "/usr/lib/python3.8/multiprocessing/managers.py", line 623, in _create
conn = self._Client(self._address, authkey=self._authkey)
File "/usr/lib/python3.8/multiprocessing/connection.py", line 502, in Client
c = SocketClient(address)
File "/usr/lib/python3.8/multiprocessing/connection.py", line 630, in SocketClient
s.connect(address)
ConnectionRefusedError: [Errno 111] Connection refused
Prior to this, the code has used pump to spawn processes (up to 50 on a server with 64 cores). There is also plenty of memory. Since pymp has worked earlier in the code, I know it can open connections (I'm not sure what port), but it has worked earlier in the code.
I have tried different input data sets, six of them of various sizes, and all have finished correctly (they match the "serial" version of the code).
Any suggestions where to continue looking? What port is used by default?
Thanks!
Jeff