Open plasorak opened 1 year ago
Transferring to iomanager
, as I think the bulk of the changes will have to be made here. ZMQ does not support hostnames in its URIs, only IPs, so some resolution must be done. However, the Connectivity Service should be given the versions of the URI that contain the hostname so that the resolution is done correctly within the K8s containers.
We checked with Tiago that passing hostname here works inside K8s (I'm assuming that the IP/hostnames are the same in K8s or that something else does it).
So basically what I understand should happen is that the boot.use_k8s
should be translated to no a turn_off_resolution
iomanager parameter around and make this possible. I'll have a look at this next week with Tiago.
So more research into ZMQ shows that I was partially incorrect...calls to bind
require IP addresses, but calls to connect
can use a hostname. This led to my current set of changes, where IOManager does not update its connection strings with resolved information by default, instead publishing to the Connectivity Service the unresolved hostnames. The exception is when a wildcard is present in the input connection string, either a port (e.g. tcp://dfo:*
), which will be allocated to an epheremal port on bind
, or a host (e.g. tcp://*:1234
), which listens on all interfaces, but should be given to Connectivity Service as one of the IPs that the socket was actually bound to. (I believe that this functionality is never used in DUNE-DAQ.)
So, just trying to hammer this home:
In cases where both SSH and connectivity service are used: daqconf here and here will create a connection of the form tcp://{dfo}:*
in the conf json. This is then passed to nanorc which converts it to tcp://np04-srv-001:*
. This then gets published in the connectivity services as tcp://np04-srv-001:40224
. The addresses given to ZMQ are tcp://10.73.136.56:*
and tcp://np04-srv-001:40224
for the receiver and sender, respectively.
In cases where SSH and no connectivity service are used: the same piece of code in daqconf will create a connection of the form tcp://{dfo}:1234
. Nanorc resolves the hostname tcp://np04-srv-001:1234
. The addresses given to ZMQ are tcp://10.73.136.56:12353
and tcp://np04-srv-004:12352
for the receiver and sender, respectively
In cases where K8s and connectivity service are used: the connection is tcp://dfo:1234
in the conf json. Nanorc doesn’t do anything to this. The published connectivity services addresses are of the form tcp://dfo:1234
. The addresses given to ZMQ are tcp://10.244.8.115:1234
and tcp://dfo:1234
for the receiver and sender, respectively. I’m a bit surprised that the receiver side works.
I haven't investigated the no k8s no connectivity service.
This line assumes that the C++ should resolve the IP address of the host. This breaks DAQ applications running in K8s
I think the code here is superseded by the connectivity service which is where we should resolve all the IPs, right?
My two cents here would be to remove this line when we have the connectivity service running in a "consolidated way."