search

JuliaLang / Distributed.jl

Create and control multiple Julia processes remotely for distributed computing. Ships as a Julia stdlib.
https://docs.julialang.org/en/v1/stdlib/Distributed/
MIT License
20 stars 8 forks source link

allow initiating peer to close stream gracefully #9

Open simeonschaub opened 8 months ago

simeonschaub commented 8 months ago

I have a usecase where I would like to remove a worker without killing it. Currently, trying to disconnect from the head node will cause the message handler loop to throw a fatal exception, so this adds a check that the connection is still open when trying to read new messages.

A minimal example of what I'm trying to do can be found here, could adapt that into a test if so desired.

simeonschaub commented 8 months ago

@jpsamaroo Looks like GitHub won't let me request a review, but I'd be curious to hear your thoughts on this. Is there a reason we might still want a closed connection to be fatal here?

jpsamaroo commented 8 months ago

This seems like a good idea to me, but I will yield to @vchuravy for further review. Until then, I'd want a test to ensure that the worker only remains alive when it's incoming to the head node. Additionally, some documentation on how to initiate this kind of connection (with an example) would be an excellent addition to the docs.

simeonschaub commented 8 months ago

Additionally, some documentation on how to initiate this kind of connection (with an example) would be an excellent addition to the docs.

Agree! My current thinking is that it might be best to move this functionality into a new package and then just link to that in the documentation. Since Distributed is just a regular package now, we should be able to then add this as a test dependency here, right?

Eventually, it might even make sense to move the implementation here but I'd like to experiment with this some more first before we commit to an official API.

vchuravy commented 8 months ago

We certainly would need a test here for this. Right now I am moving slowly on Distributed.jl since I have my hands full with the rest of the stdlibs and I want to move cautiously until we have 1.11 registered.