pyiron / pylammpsmpi

Parallel Lammps Python interface - control a mpi4py parallel LAMMPS instance from a serial python process or a Jupyter notebook
https://pylammpsmpi.readthedocs.io
BSD 3-Clause "New" or "Revised" License
30 stars 4 forks source link

socket connection stuck #22

Closed srmnitc closed 4 years ago

srmnitc commented 4 years ago

When using socket as communicator, the creation of the object seems to get stuck on hostsocket, _ = s.accept() - Line 85 from communicate.py

jan-janssen commented 4 years ago

Yes, I did not have the time to test it yet.

jan-janssen commented 4 years ago

Btw. Are you at ICAMS today?

srmnitc commented 4 years ago

I am not now, I am working from home nowadays. But I will come around 13.00 - I have meetings from 14.00

srmnitc commented 4 years ago

Are you around?

jan-janssen commented 4 years ago

We have a PhD defense today, that is why I am at ICAMS, but I do not think I am going to stay till 13:00 - so maybe it is easier to meet another day.

srmnitc commented 4 years ago

Yeah, I do not think I will be able to make it before that. Let's meet another day then!

jan-janssen commented 4 years ago

Ok, I get the point why it is not working - it is basically waiting for the connection of the client before it continuous.

jan-janssen commented 4 years ago

I tired to separate the connect part - https://github.com/pyiron/pylammpsmpi/commit/1f25daef8a7b4c997de38dcc4f8719cdf73a22c0

srmnitc commented 4 years ago

A little bit busy with some other things - I will contribute more soon! sorry!

jan-janssen commented 4 years ago

In terms of using higher level libraries and not reinventing the wheel, we should consider using dask for socket communication: https://distributed.dask.org/en/latest/communications.html

srmnitc commented 4 years ago

Yeah, we could play around with it. I use dask quite a bit in my normal work. This might also be of interest. https://distributed.dask.org/en/latest/actors.html

On Tue, Mar 31, 2020, 07:59 Jan Janssen notifications@github.com wrote:

In terms of using higher level libraries and not reinventing the wheel, we should consider using dask for socket communication: https://distributed.dask.org/en/latest/communications.html

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/pyiron/pylammpsmpi/issues/22#issuecomment-606417609, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADAKESM2WDOIXR6S3CTFV6DRKGBENANCNFSM4LE3EL6Q .

srmnitc commented 4 years ago

@jan-janssen A combination of dask-distributed and dask-jobqueue looks promising. I have started a new branch. It is all clunky at the moment, but when things shape up I will update you.

jan-janssen commented 4 years ago

This might even be interesting beyond the Lammps interface, like having a generic interface for connecting to calculators via sockets.

srmnitc commented 4 years ago

Yeah, I am super excited to get this working. I went for two workshops organised by ECAM in which they covered dask. I briefly discussed with one of the guys here at Julich. If we need more help, that would also be possible I think. Let us see how it goes.

srmnitc commented 4 years ago

32 is almost ready. Using jobqueue and dask actors, we can run pylammpmpi on queuing systems while being able to interact with it directly from a jupyter notebook. The code has some restructuring, a higher level wrapper is added to convert dask futures to actual values. It also works with the local mode. I have added some example in the form of notebooks in the notebooks folder. One for local cluster and one for slurm. dask-distributed and dask-jobqueue should also be added to dependencies. I have not yet tested extensively, I will be running a couple of projects later on to see how it works out. I think its really cool :)

srmnitc commented 4 years ago

Of course this issue would then be closed if we merge #32