search

Parsl / parsl

Parsl - a Python parallel scripting library
http://parsl-project.org
Apache License 2.0
498 stars 195 forks source link

SSH based channels assume a completely open network, but most networks are not #2601

Open astro-friedel opened 1 year ago

astro-friedel commented 1 year ago

When trying to use SSHInteractiveLoginChannel to submit Parsl scripts remotely I run into issues when I am on a network that has any type of firewall. I can get the ssh connection initiated and the process_worker_pool.py starts up, but quickly fails with the message

Failed to find a viable address to connect to interchange. Exiting

I was able to trace this to zmq attempting a tcp connection from the remote machine to port 54134 on my machine and the network firewall blocking it. Most networks have a lot of blocked ports, so this reduces the usability of these channel types.

Describe the solution you'd like It would be nice if the initial ssh connection could be used to tunnel the zmq traffic between my local machine and the remote one. This would eliminate the issue of zmq trying to connect to blocked ports and also allow for the use of these channels from behind a NAT.

Describe alternatives you've considered I have considered initiating an ssh tunnel from a separate process before running the parsl code, but this breaks the nice enclosed model in which parsl works.

benclifford commented 3 months ago

pleases see issue #3515 about removing channels from Parsl