Parsl / parsl

Parsl - a Python parallel scripting library
http://parsl-project.org
Apache License 2.0
484 stars 194 forks source link

sometimes Interchange will accept a hostname address, in violation of test expectations #3037

Open benclifford opened 7 months ago

benclifford commented 7 months ago

Describe the bug In PR #2828, interchange binding behavior was changed with the expectation that it would no longer accept a hostname as a binding address.

This no-hostname behaviour is tested in parsl/tests/test_htex/test_htex_zmq_binding.py, test_interchange_binding_with_non_ipv4_address which tests that starting the interchange will raise a ZMQError when the interchange is passed the address localhost.

In an install I just made on Perlmutter, this test fails: the interchange will accept localhost as an address, and can successfully run the test suite with that change:

--- a/parsl/tests/configs/htex_local_alternate.py
+++ b/parsl/tests/configs/htex_local_alternate.py
@@ -39,7 +39,7 @@ def fresh_config():
     return Config(
         executors=[
             HighThroughputExecutor(
-                address="127.0.0.1",
+                address="localhost",
                 label="htex_Local",
                 working_dir=working_dir,
                 storage_access=[FTPInTaskStaging(), HTTPInTaskStaging(), NoOpFileStaging()],

I have tried on my laptop and the same change results in the expected hang, with this reported in interchange.log:

  File "/home/benc/parsl/virtualenv-3.11/lib/python3.11/site-packages/zmq/sugar/socket.py", line 498, in bind_to_random_port
    self.bind(f'{addr}:{port}')
  File "/home/benc/parsl/virtualenv-3.11/lib/python3.11/site-packages/zmq/sugar/socket.py", line 302, in bind
    super().bind(addr)
  File "zmq/backend/cython/socket.pyx", line 564, in zmq.backend.cython.socket.Socket.bind
  File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: No such device (addr='tcp://localhost:54144')

On perlmutter, it looks like libzmq is coming through conda:

conda list | grep -i zero zeromq 4.3.5 h59595ed_0 conda-forge

/global/homes/b/bxc/.conda/envs/tmp-parsl-slurm-test/lib/libzmq.so.5.2.5

(the .so version is different from the release version in ZMQ, as far as I understand things).

On my laptop dev environment, libzmq is coming via I think a binary install from pip install pyzmq - there's no conda involved in that virtualenv:

/home/benc/parsl/virtualenv-3.11/lib/python3.11/site-packages/pyzmq.libs/libzmq-f468291a.so.5.2.4

To Reproduce

?

with 6c1d4df65ea1eba58839b02a5b22b93394ae2bc1 release 2024.01.29

Expected behavior

If zmq will sometimes accept a hostname address in some situations, maybe this test should be removed.

Alternatively, if forcing an address to be IPv4 format is what is desired, then some different error handling should happen outwith pyzmq/libzmq

benclifford commented 5 months ago

This is now failing in CI repeatedly for me. I'll disable that test to unblock CI.

@yadudoc introduced that test in c39700bc2283500e0327ed3d7f00cf0056f2dfed and I'll assign this issue to him for un-disabling/fixing.

benclifford commented 5 months ago

3360 removes one of the tests introduced as part of security work, so may now be permitting unintended security vulnerabilities.