search

Amber-MD / pytraj

Python interface of cpptraj
https://amber-md.github.io/pytraj
172 stars 38 forks source link

pmap parallel error #1651

Open groponp opened 8 months ago

groponp commented 8 months ago

Hi,

I'm trying to use the pmap method : In [7]: pt.version Out[7]: '2.0.5'

traj = pt.iterload("04_MD/MD_r1.dcd", "files/MD_QwikMD.psf") pt.pmap(pt.radgyr, traj, n_cores=4)

error:

TypeError Traceback (most recent call last)

in ----> 1 pt.pmap(pt.radgyr, traj, n_cores=4) ~/miniconda3/envs/cpptraj/lib/python3.7/site-packages/pytraj/parallel/multiprocess.py in pmap(func, traj, *args, **kwargs) 228 def pmap(func=None, traj=None, *args, **kwargs): 229 if func != NH_order_parameters: --> 230 return _pmap(func, traj, *args, **kwargs) 231 else: 232 if 'n_cores' in kwargs.keys(): ~/miniconda3/envs/cpptraj/lib/python3.7/site-packages/pytraj/parallel/multiprocess.py in _pmap(func, traj, *args, **kwargs) 218 progress_params=progress_params) 219 --> 220 data = p.map(pfuncs, [rank for rank in range(n_cores)]) 221 p.close() 222 ~/miniconda3/envs/cpptraj/lib/python3.7/multiprocessing/pool.py in map(self, func, iterable, chunksize) 266 in a list that is returned. 267 ''' --> 268 return self._map_async(func, iterable, mapstar, chunksize).get() 269 270 def starmap(self, func, iterable, chunksize=None): ~/miniconda3/envs/cpptraj/lib/python3.7/multiprocessing/pool.py in get(self, timeout) 655 return self._value 656 else: --> 657 raise self._value 658 659 def _set(self, i, obj): ~/miniconda3/envs/cpptraj/lib/python3.7/multiprocessing/pool.py in _handle_tasks(taskqueue, put, outqueue, pool, cache) 429 break 430 try: --> 431 put(task) 432 except Exception as e: 433 job, idx = task[:2] ~/miniconda3/envs/cpptraj/lib/python3.7/multiprocessing/connection.py in send(self, obj) 204 self._check_closed() 205 self._check_writable() --> 206 self._send_bytes(_ForkingPickler.dumps(obj)) 207 208 def recv_bytes(self, maxlength=None): ~/miniconda3/envs/cpptraj/lib/python3.7/multiprocessing/reduction.py in dumps(cls, obj, protocol) 49 def dumps(cls, obj, protocol=None): 50 buf = io.BytesIO() ---> 51 cls(buf, protocol).dump(obj) 52 return buf.getbuffer() 53 ~/miniconda3/envs/cpptraj/lib/python3.7/site-packages/pytraj/trajectory/c_traj/c_trajectory.cpython-37m-x86_64-linux-gnu.so in pytraj.trajectory.c_traj.c_trajectory.TrajectoryCpptraj.__reduce_cython__() TypeError: no default __reduce__ due to non-trivial __cinit__
hainm commented 8 months ago

it seems related to cython with python version. @groponp Can you please try pytraj with python 3.11 instead?

conda install conda-forge::ambertools

groponp commented 7 months ago

Dear @hainm, thanks so much for your answer !

I would can try it.

groponp commented 7 months ago

@hainm I want to asked you, too. Is autoimage enable to use him with pmap?. Now when I use pmap, Have always I that contrast against serial calculation ?

hainm commented 7 months ago

@hainm I want to asked you, too. Is autoimage enable to use him with pmap?. Now when I use pmap, Have always I that contrast against serial calculation ?

Dear @groponp , what do you mean about "him" in this question?

groponp commented 7 months ago

I'm sorry I have a mistake , I want to say use with pmap,

hainm commented 7 months ago

I'm sorry I have a mistake , I want to say use with pmap,

@groponp I don't think so. pmap only works with method having register_map decorator: https://github.com/Amber-MD/pytraj/blob/a318898a88d3506305d2e409ccd1344e452b2a40/pytraj/all_actions.py