Add multiplatform compatibility (tested on macOS 12)

[belotv]

Replaced Avahi by DNS-SD/MDNS wrapper (https://github.com/mjansson/mdns) compatible with all OS Support IPV4 and IPV6 Removed unused files Do not try to free session[device].data if not connected

[lundmar]

Looks good, except please do not remove src/vxi11core_svc.c . I'm keeping this around because I may use it in the future if I get to adding server VXI11 functionality to liblxi.

[lundmar]

Great, thanks for your contributions.

I'm glad your patches get rid of the Avahi dependency in favour of something simpler and cross platform and it is also going to be useful to not be dependent on the Avahi server which may or may not be installed/enabled as I have recently experienced trying to create a flatpak. Hopefully it will perform just as well as Avahi. Of course, it is also the first step to support IPv6 which is eventually going to happen.

Looking forward to some screenshots of lxi-tools running on MacOs ;)

[lundmar]

Just tested, there are a few issues:

1. It only finds 1 out of my 3 mDNS enabled instruments
2. It's calling the broadcast callback which it should not (only meant for standard discovery)

I'll look into these issues when I get time.

[belotv]

1. For me it finds the same instruments as the native mdns service from OSX. I am not sure at which extent Avahi goes above the standard. Can you run dns-sd -B _vxi-11._tcp and tell me what is the output ?

2. Ok, I thought it would be interesting to provide info on which interfaces sockets got opened for listening so I used the callback, but it can easily be commented out.

[lundmar]

On Linux we do not have dns-sd but the avahi equivalent.

I would like to see the mdns implementation at least find the same as avahi:

lundmar@wopr:~$ avahi-browse -t _vxi-11._tcp
+ enxe4b97a86fdad IPv4 KIKUSUI PMX35-3A DC Power Supply - XL000024   _vxi-11._tcp         local
+ enxe4b97a86fdad IPv4 Rohde&Schwarz RTB2004 - 000000                _vxi-11._tcp         local
lundmar@wopr:~$ avahi-browse -t _scpi-raw._tcp
+ enxe4b97a86fdad IPv6 Rohde&Schwarz power supply NGM202-101403      _scpi-raw._tcp       local
+ enxe4b97a86fdad IPv4 KIKUSUI PMX35-3A DC Power Supply - XL000024   _scpi-raw._tcp       local
+ enxe4b97a86fdad IPv4 Rohde&Schwarz RTB2004 - 000000                _scpi-raw._tcp       local
+ enxe4b97a86fdad IPv4 Rohde&Schwarz power supply NGM202-101403      _scpi-raw._tcp       local
lundmar@wopr:~$ avahi-browse -t _lxi._tcp
+ enxe4b97a86fdad IPv6 Rohde&Schwarz power supply NGM202-101403      _lxi._tcp            local
+ enxe4b97a86fdad IPv4 KIKUSUI PMX35-3A DC Power Supply - XL000024   _lxi._tcp            local
+ enxe4b97a86fdad IPv4 Rohde&Schwarz power supply NGM202-101403      _lxi._tcp            local

That's 3 different instruments, various instrument related services.

It's okay it is calling the broadcast function - it kind of serves it's purpose when using the command-line version but using lxi-gui the broadcast calls are fired all at once so it basically only shows the user the last broadcast interface because of message overlap. While this makes for quick discovery I wouldn't mind maybe doing the mDNS multicast one interface at a time much like how the standard discovery does it. This way the broadcast information will show better in the GUI. We may even introduce a multicast callback instead of reusing the broadcast callback. This way we can also more precisely say "Multicasting on interface ...".

[lundmar]

My lxi output:

$ lxi discover -m
Searching for LXI devices - please wait...

Broadcasting on interface enxe4b97a86fdad
Broadcasting on interface virbr0
Broadcasting on interface mpqemubr0
  Found "Rohde&Schwarz power supply NGM202-101403" on address 192.168.0.107
    lxi service on port 80
  Found "Rohde&Schwarz power supply NGM202-101403" on address 192.168.0.107
    scpi-raw service on port 5025

Found 2 services

[lundmar]

Going back to the mdns library repository, I tried compiling their example discovery app, same result - only finds the one instrument:

lundmar@wopr:~/test/git/mdns/bin$ ./mdns_example --query _vxi-11._tcp.local.
Local IPv4 address: 192.168.0.125
Local IPv4 address: 192.168.122.1
Local IPv4 address: 10.65.19.1
Opened 3 sockets for mDNS query
Sending mDNS query : _vxi-11._tcp.local. PTR
Reading mDNS query replies
Read 0 records
Closed sockets

lundmar@wopr:~/test/git/mdns/bin$ ./mdns_example --query _scpi-raw._tcp.local.
Local IPv4 address: 192.168.0.125
Local IPv4 address: 192.168.122.1
Local IPv4 address: 10.65.19.1
Opened 3 sockets for mDNS query
Sending mDNS query : _scpi-raw._tcp.local. PTR
Reading mDNS query replies
192.168.0.107:5353 : answer _scpi-raw._tcp.local. PTR Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local. rclass 0x1 ttl 10 length 43
192.168.0.107:5353 : additional Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local. SRV rs-instrument.local. priority 0 weight 0 port 5025
192.168.0.107:5353 : additional Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local. TXT Manufacturer = Rohde&Schwarz
192.168.0.107:5353 : additional Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local. TXT Model = NGM202
192.168.0.107:5353 : additional Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local. TXT SerialNumber = 101403
192.168.0.107:5353 : additional Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local. TXT FirmwareVersion = 03.068 00A8F863604
192.168.0.107:5353 : additional rs-instrument.local. A 192.168.0.107
Read 4 records
Closed sockets

lundmar@wopr:~/test/git/mdns/bin$ ./mdns_example --query _lxi._tcp.local.
Local IPv4 address: 192.168.0.125
Local IPv4 address: 192.168.122.1
Local IPv4 address: 10.65.19.1
Opened 3 sockets for mDNS query
Sending mDNS query : _lxi._tcp.local. PTR
Reading mDNS query replies
192.168.0.107:5353 : answer _lxi._tcp.local. PTR Rohde&Schwarz power supply NGM202-101403._lxi._tcp.local. rclass 0x1 ttl 10 length 43
192.168.0.107:5353 : additional Rohde&Schwarz power supply NGM202-101403._lxi._tcp.local. SRV rs-instrument.local. priority 0 weight 0 port 80
192.168.0.107:5353 : additional Rohde&Schwarz power supply NGM202-101403._lxi._tcp.local. TXT Manufacturer = Rohde&Schwarz
192.168.0.107:5353 : additional Rohde&Schwarz power supply NGM202-101403._lxi._tcp.local. TXT Model = NGM202
192.168.0.107:5353 : additional Rohde&Schwarz power supply NGM202-101403._lxi._tcp.local. TXT SerialNumber = 101403
192.168.0.107:5353 : additional Rohde&Schwarz power supply NGM202-101403._lxi._tcp.local. TXT FirmwareVersion = 03.068 00A8F863604
192.168.0.107:5353 : additional rs-instrument.local. A 192.168.0.107
Read 4 records
Closed sockets

[belotv]

Without having any undetected device it is hard to help you debug. I will have a look at avahi source to see if there is anything special they are doing. Have you tried to physically disconnect the Rohde&Schwarz power supply NGM202-101403 to see if the other get discovered ? Longer timeout does not work either ?

[belotv]

Maybe you should try to add some debug printf in mdns_discovery_recv. Are your devices RFC 6762 compliant ?

// According to RFC 6762 the query ID MUST match the sent query ID (which is 0 in our case)
    if (query_id || (flags != 0x8400))
        return 0;  // Not a reply to our question

    // It seems some implementations do not fill the correct questions field,
    // so ignore this check for now and only validate answer string
    // if (questions != 1)
    //  return 0;
etc....

There are a lot of cases returning 0 in mdns_discovery_recv, (called when running --discover), hopefully answer is received but discarded - in that case that may be easy to fix. If it is not received at all, it may be linked to the query sent.

Maybe modifying the query QCLASS to MDNS_CLASS_ANY rather than MDNS_CLASS_IN (by modifying mdns_services_query constant, and updating mdns_discovery_recv to ensure all answers are logged) ?

avahi-daemon has sadly no option to dump the raw query or answers ... that would be much easier to solve the issue.

[lundmar]

There may be a lot of instruments out there colouring outside the lines of the spec. However, if Avahi can detect them I would like to make it so that mdns can detect them too. I don't want to miss out on detecting these instruments regardless of their maybe flawed implementation because I suspect it may be a lot of them.

[lundmar]

When we do figure out how to fix this and if this is the instruments breaking spec then we could have an option to use strict mDNS discovery - would be useful for manufacturers testing their new instrument implementation.

[belotv]

Fully agree, I suggest you revert to avahi on master and create a specific branch with mdns until we find the root cause (unless you find it tonight).

[lundmar]

It's okay - it can stay on master for now as this will be my focus. I'll move it to a branch if I need to apply other changes.

[lundmar]

I haven't had time to start debugging yet because there has been a lot of activity surrounding lxi-tools in forums, PRs, etc. that I needed to respond to.

Also, lxi-tools just got a mention in https://thisweek.gnome.org and I was reading it and saw the news that gstreamer had included their plugins in their main repository. So curious as I am I went looking to see if they were kidding and it would just be git submodules but no, they actually merged everything into one giant code base. I think it is the correct move - then everything is a matching set.

Anyway, long story short. Going through the gstreamer repo I accidentally stumbled across one of their meson wrappers named libmicrodns.wrap - apparently there is another C cross-platform mDNS effort: https://github.com/videolabs/libmicrodns

Of course I had to give it a quick test. It includes a quite simple listen example which is easy to modify. I ran it and it actually provides the exact same results as Avahi for all the various service names.

In short, we have another option in case the current mdns library proves too cumbersome to fix. I haven't looked at their source so I don't know which is the better project to bet on.

[lundmar]

I'm getting a whole lot of raw data from libmicrodns query responses. This one includes response from all 3 devices when querying for _scpi-raw service:

lundmar@wopr:~/test/git/libmicrodns$ ./build/examples/listen
[{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"Rohde&Schwarz RTB2004 - 000000._scpi-raw._tcp.local"}}]
[{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"UNKNOWN","class":"IN","data":null},{"name":"rs-instrument.local","type":"UNKNOWN","class":"IN","data":null},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"SRV","class":"IN","data":{"target":"rs-instrument.local","port":5025,"priority":0,"weight":0}},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"TXT","class":"IN","data":{"text":["FirmwareVersion=03.068 00A8F863604","SerialNumber=101403","Model=NGM202","Manufacturer=Rohde&Schwarz"]}},{"name":"rs-instrument.local","type":"A","class":"IN","data":{"address":"192.168.0.107"}},{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local"}}]
[{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"UNKNOWN","class":"IN","data":null},{"name":"rs-instrument.local","type":"UNKNOWN","class":"IN","data":null},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"SRV","class":"IN","data":{"target":"rs-instrument.local","port":5025,"priority":0,"weight":0}},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"TXT","class":"IN","data":{"text":["FirmwareVersion=03.068 00A8F863604","SerialNumber=101403","Model=NGM202","Manufacturer=Rohde&Schwarz"]}},{"name":"rs-instrument.local","type":"A","class":"IN","data":{"address":"192.168.0.107"}},{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local"}}]
[{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"UNKNOWN","class":"IN","data":null},{"name":"rs-instrument.local","type":"UNKNOWN","class":"IN","data":null},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"SRV","class":"IN","data":{"target":"rs-instrument.local","port":5025,"priority":0,"weight":0}},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"TXT","class":"IN","data":{"text":["FirmwareVersion=03.068 00A8F863604","SerialNumber=101403","Model=NGM202","Manufacturer=Rohde&Schwarz"]}},{"name":"rs-instrument.local","type":"A","class":"IN","data":{"address":"192.168.0.107"}},{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local"}}]
[{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"UNKNOWN","class":"IN","data":null},{"name":"rs-instrument.local","type":"UNKNOWN","class":"IN","data":null},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"SRV","class":"IN","data":{"target":"rs-instrument.local","port":5025,"priority":0,"weight":0}},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"TXT","class":"IN","data":{"text":["FirmwareVersion=03.068 00A8F863604","SerialNumber=101403","Model=NGM202","Manufacturer=Rohde&Schwarz"]}},{"name":"rs-instrument.local","type":"A","class":"IN","data":{"address":"192.168.0.107"}},{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local"}}]
[{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"UNKNOWN","class":"IN","data":null},{"name":"rs-instrument.local","type":"UNKNOWN","class":"IN","data":null},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"SRV","class":"IN","data":{"target":"rs-instrument.local","port":5025,"priority":0,"weight":0}},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"TXT","class":"IN","data":{"text":["FirmwareVersion=03.068 00A8F863604","SerialNumber=101403","Model=NGM202","Manufacturer=Rohde&Schwarz"]}},{"name":"rs-instrument.local","type":"A","class":"IN","data":{"address":"192.168.0.107"}},{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local"}}]
[{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"UNKNOWN","class":"IN","data":null},{"name":"rs-instrument.local","type":"UNKNOWN","class":"IN","data":null},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"SRV","class":"IN","data":{"target":"rs-instrument.local","port":5025,"priority":0,"weight":0}},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"TXT","class":"IN","data":{"text":["FirmwareVersion=03.068 00A8F863604","SerialNumber=101403","Model=NGM202","Manufacturer=Rohde&Schwarz"]}},{"name":"rs-instrument.local","type":"A","class":"IN","data":{"address":"192.168.0.107"}},{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local"}}]
[{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"UNKNOWN","class":"IN","data":null},{"name":"rs-instrument.local","type":"UNKNOWN","class":"IN","data":null},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"SRV","class":"IN","data":{"target":"rs-instrument.local","port":5025,"priority":0,"weight":0}},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"TXT","class":"IN","data":{"text":["FirmwareVersion=03.068 00A8F863604","SerialNumber=101403","Model=NGM202","Manufacturer=Rohde&Schwarz"]}},{"name":"rs-instrument.local","type":"A","class":"IN","data":{"address":"192.168.0.107"}},{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local"}}]
[{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"UNKNOWN","class":"IN","data":null},{"name":"rs-instrument.local","type":"UNKNOWN","class":"IN","data":null},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"SRV","class":"IN","data":{"target":"rs-instrument.local","port":5025,"priority":0,"weight":0}},{"name":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local","type":"TXT","class":"IN","data":{"text":["FirmwareVersion=03.068 00A8F863604","SerialNumber=101403","Model=NGM202","Manufacturer=Rohde&Schwarz"]}},{"name":"rs-instrument.local","type":"A","class":"IN","data":{"address":"192.168.0.107"}},{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"Rohde&Schwarz power supply NGM202-101403._scpi-raw._tcp.local"}}]
[{"name":"KIKUSUI PMX35-3A DC Power Supply - XL000024._scpi-raw._tcp.local","type":"TXT","class":"IN","data":{"text":["FirmwareVersion=IFC01.52.0011 IOC01.10.0069","SerialNumber=XL000024","Model=PMX35-3A","Manufacturer=KIKUSUI","txtvers=1"]}},{"name":"KIKUSUI PMX35-3A DC Power Supply - XL000024._scpi-raw._tcp.local","type":"SRV","class":"IN","data":{"target":"PMX35-3A-00024.local","port":5025,"priority":0,"weight":0}},{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"KIKUSUI PMX35-3A DC Power Supply - XL000024._scpi-raw._tcp.local"}}]
[{"name":"KIKUSUI PMX35-3A DC Power Supply - XL000024._scpi-raw._tcp.local","type":"TXT","class":"IN","data":{"text":["FirmwareVersion=IFC01.52.0011 IOC01.10.0069","SerialNumber=XL000024","Model=PMX35-3A","Manufacturer=KIKUSUI","txtvers=1"]}},{"name":"KIKUSUI PMX35-3A DC Power Supply - XL000024._scpi-raw._tcp.local","type":"SRV","class":"IN","data":{"target":"PMX35-3A-00024.local","port":5025,"priority":0,"weight":0}},{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"KIKUSUI PMX35-3A DC Power Supply - XL000024._scpi-raw._tcp.local"}}]
[{"name":"KIKUSUI PMX35-3A DC Power Supply - XL000024._scpi-raw._tcp.local","type":"TXT","class":"IN","data":{"text":["FirmwareVersion=IFC01.52.0011 IOC01.10.0069","SerialNumber=XL000024","Model=PMX35-3A","Manufacturer=KIKUSUI","txtvers=1"]}},{"name":"KIKUSUI PMX35-3A DC Power Supply - XL000024._scpi-raw._tcp.local","type":"SRV","class":"IN","data":{"target":"PMX35-3A-00024.local","port":5025,"priority":0,"weight":0}},{"name":"_scpi-raw._tcp.local","type":"PTR","class":"IN","data":{"domain":"KIKUSUI PMX35-3A DC Power Supply - XL000024._scpi-raw._tcp.local"}}]

[lundmar]

Here is the output of wireshark when running "lxi discover -m". All 3 devices responds but lxi only prints services lxi and scpi-raw found for NGM202-101403. https://file.io/GZgp1h8JCKPF

[lundmar]

Quickly inspecting the response messages, I bet it has something to do with the "Questions" value in the MDNS responses. The one device which is picked up, the NGM202 (IP 192.168.0.107), answers with questions=1, the rest answers with questions=0.

[lundmar]

So, I was debugging myself through the mdns code to inspect the active code path during message receive simply to find out there was no activity at all except for the response messages from NGM202. So I went back to the wireshark trace and realized that the problem must be that the mdns process sends/multicasts from an arbitrary port (in the trace, port 48337) to port 5353 and the NGM202 device answers responses successfully back to this sender port. However, the responses of the other devices are sent to back to port 5353 and simply lost because the mdns process clearly does not listen to that.

I don't know the details of the MDNS protocol but I suspect legacy MDNS probably expects to send to and receive on port 5353 and hence the failure to communicate. So the fix may be to simply reconfigure the mdns library to send from and listen to fixed port 5353.

[lundmar]

Actually, it is not just the port. The NGM202 device successfully sends it's response back to sender IP:port, while the ignored devices sends back their responses to 224.0.0.251:5353, meaning they are actually multi-casting their responses back.

[belotv]

The behaviour of the undetected devices seems to be compliant.

I am reading the RFC and there are two cases for mdns queries, one-shot multicast query, that should not use 5353 and client can answer blindly to 224.0.0.251:5353 and continuous one, which should use 5353. Maybe changing the port will be sufficient. Did you try yet ?

[belotv]

If not working, I would not spend more time trying to fix this mdns implementation and I would just move to libmicrodns which seems to work fine, that’s a great discovery :)

But according to the mdns documentation switching to 5353 should work. It is clearly stated in the doc that when using ephemeral port it requests a unicast response and discards the multicast ones, whereas when set to 5353 it will request and process multicast response.

[lundmar]

I'm afraid we struck out of luck. The mdns library does not support continuos queries :(

The answers why are here: https://github.com/mjansson/mdns/issues/25 https://github.com/mjansson/mdns/issues/13

I'm afraid the only path forward is to port your work to libmicrodns... Is this something you would like to give a shot?

[belotv]

No problem, libmicrodns seems to be quite straightforward to implement, I will most likely do that tomorrow.

[lundmar]

Great, libmicrodns does seem like a nice implementation and there is some activity in the repo so it is not a dead project.

I will be rude and rewrite the git history to before this merge.

[lundmar]

Oh, by the way. Feel free to create a multicast callback similar to the broadcast callback so that we can print the progress in the console so it will look something like this:

Multicasting on interface ...

[belotv]

Ok, for info I will not be able to use the library as-is as it does not work on macOS due to the fact that when a single interface is not available, it prevents any interface to be used (mdns_init function is quite severe). And unfortunately the mdns_resolve function being static, I cannot simply write another init function within lxi, so I guess I will have to fork the library (will do a pull request on the library itself).

The idea will be to remove the unavailable interfaces from ctx rather than completely destroying the context.

[lundmar]

I see, I'm sure the microdns author will appreciate changes to make it run on macos. After all, it claims to be cross platform friendly.

[belotv]

BTW, did you try to use mdns with port 5353 at the end ? From what I understand from the author, the issue 13/25 you reported are from March 2020, but it seems this has been fixed in Apr/Jun, and now the library is indeed able to receive multicast response as long as we use port 5353. Since I have no unrecognised instrument I cannot test myself though.

I will still port libmicrodns, but would be good if we have two alternatives.

[lundmar]

No, I didn't try that. Do you know what to reconfigure in the mdns library or API to do that?

I assumed that was part of the continuous feature.

[belotv]

Port is the last parameter of the "open_client_sockets" function, so simply replace 0 by 5353 when that function is called in mdns.c

[lundmar]

Right, so now it discovers one additional instrument, my KIKUSUI psu, however, the output is duplicated multiple times and there is something wrong with the strings:

Searching for LXI devices - please wait...

Broadcasting on interface enxe4b97a86fdad
Broadcasting on interface virbr0
Broadcasting on interface mpqemubr0
  Found "Rohde&Schwarz power supply NGM202-101403" on address 192.168.0.107
    lxi service on port 80
  Found "KIKUSUI PMX35-3A DC supply NGM202-101403" on address 192.168.0.117
    lxi service on port 80
  Found "KIKUSUI PMX35-3A DC Powery NGM202-101403" on address 192.168.0.117
    vxi-11 service on port 5025
  Found "KIKUSUI PMX35-3A DC Powery NGM202-1014" on address 192.168.0.117
    scpi-raw service on port 80
  Found "KIKUSUI PMX35-3A DC Powery NGM202-1014" on address 192.168.0.117
    hislip service on port 111
  Found "KIKUSUI PMX35-3A DC Power SNGM202-1014" on address 192.168.0.117
    scpi-telnet service on port 5025
  Found "KIKUSUI PMX35-3A DC Power SNGM202-1014" on address 192.168.0.117
    lxi service on port 80
  Found "KIKUSUI PMX35-3A DC Power SNGM202-1014" on address 192.168.0.117
    vxi-11 service on port 5025
  Found "KIKUSUI PMX35-3A DC Power SNGM202-1014" on address 192.168.0.117
    scpi-raw service on port 80
  Found "KIKUSUI PMX35-3A DC Power SNGM202-1014" on address 192.168.0.117
    hislip service on port 111
  Found "KIKUSUI PMX35-3A DC Power SNGM202-1014" on address 192.168.0.117
    scpi-telnet service on port 5025
  Found "KIKUSUI PMX35-3A DC Power SNGM202-1014" on address 192.168.0.117
    lxi service on port 80
  Found "KIKUSUI PMX35-3A DC Power SNGM202-1014" on address 192.168.0.117
    vxi-11 service on port 5025
  Found "KIKUSUI PMX35-3A DC Power SNGM202-1014" on address 192.168.0.117
    scpi-raw service on port 80
  Found "KIKUSUI PMX35-3A DC Power SNGM202-1014" on address 192.168.0.117
    hislip service on port 111
  Found "KIKUSUI PMX35-3A DC Power SNGM202-1014" on address 192.168.0.117
    scpi-telnet service on port 5025

Found 16 services

No detection of my RTB2004.

[lundmar]

I'll let you decide which path you wanna take. I mean, we can try do more debugging with your current mdns implementation if you want.

[belotv]

String issue should be easily fixable, most likely an array not resized. For the moment, I had no luck with the default listen example of libmicrodns on macOS for the moment ... Basically, the sockets cannot get opened. I am a bit surprised that even the default example does not work on macOS as the library is being used in VLC. I need a bit more time to investigate.

[lundmar]

Hmm, I would have thought the socket api would work the same on most operating systems but maybe some socket options are not supported on macOS.

Take your time, we are not in a hurry. It'll be ready when it's ready.

[belotv]

On macOS you already have Bonjour running so it may be one of the reason :/ and turns out that although it is advertised to be cross platform, libmicrodns does not actually work on macOS: https://code.videolan.org/jbk/libndi/-/issues/2

I will think about the next steps next week end.

[lundmar]

That's surprising. I assume that means that both libmicrodns and the other mdns C library conflicts with Avahi/Bonjour. If that is the case, probably the safest solution would be to continue use Avahi on Linux and introduce Bonjour on macOS.

[lundmar]

I think it is correct to assume that any alternative mDNS implementation, like our C library implementations, will conflict with avahi/bonjour servers listening on port 5353. A port necessary for some instruments to work with mDNS. I think our only choice is to use Avahi on Linux and Bonjour on macOS.

[belotv]

Thanks to SO_REUSEPORT, it is actually possible for two apps to listen on the same port (https://lwn.net/Articles/542629/), and the mdns library actually works on 5353 on macOS (and also on Linux as you were at least discovering 2 instruments out of 3), whereas libmicrodns cannot open socket on any port at all. It turns out the root cause for libmicrodns not working on macOS is that sin_len is not being defined (that library is definitely not cross-platform). So I am quite optimistic about a resolution soon.

[lundmar]

Yes, but the SO_REUSEPORT only works if avahi/bonjour also use the SO_REUSEPORT flag. I don't know if that is safe to assume.

[belotv]

This is a "SHOULD" and not a must in the RFC6762, but it is definitely implemented in Bonjour. It is also default in Avahi since 2013, so I think it is rather safe to assume.

[lundmar]

Okay, so strictly speaking it could be a problem but in practice probably most mDNS implementations use this port option as advised in the RFC. I'll accept that.

[belotv]

After checking both implementation, mdns looks much cleaner than libmicrodns, which honestly is not really an example of best practices even if maintained. And although I was able to finally run libmicrodns on macOS, it does not discover the devices which are not on the LAN from the main interface so looks like a lot of fixing is required...

With mdns header library, RBT2004 is not being shown because it does not return a SRV record according to your Wireshark extract. Basically the client is informed that the device provides a service - but the device does not tell on which port the service is running. SRV records are strongly recommended in RFC6763 but not enforced, so we should handle the case where they are missing and only a PTR record is returned.

What would you expect the callback to return as "port" in that case ? -1 ? 0 ? Default port for that type of service ? Not sure what avahi returns in that case.

[lundmar]

Avahi reports the correct port number so I did some more wireshark debugging. One of the advantages/disadvantages of avahi is that it's server is always maintaining and sometimes even caching the state of the current network so you can get a response even quicker. This also means sometimes it simply returns cached results without communication on the network. I have to restart the avahi daemon to flush its cached records to trigger and catch the full mDNS communication in wireshark and here we see that RTB2004 (192.168.0.157) does return the PTR records but subsequently also the SRV records including the port numbers.

Here is the wireshark trace: https://file.io/6WzUHnJzxvEP

[belotv]

Ok, I fully refactored my code to have something much cleaner. Could you please try the basic mdns example, changing 2 little things: -> mdns.h, line 871 - replace 0x80 by 0x00 to request QM instead of QU -> mdns.c, line 705 - replace the last "0" by MDNS_PORT I would expect it to detect all 3 devices.

[lundmar]

It's finding 2 out of 3 devices:

lundmar@wopr:~/test/git/mdns/bin$ ./mdns_example 
Local IPv4 address: 192.168.0.125:5353
Local IPv4 address: 192.168.122.1:5353
Local IPv4 address: 10.65.19.1:5353
Opened 3 sockets for DNS-SD
Sending DNS-SD discovery
Reading DNS-SD replies
192.168.0.107:5353 : answer _services._dns-sd._udp.local. PTR _http._tcp.local. rclass 0x1 ttl 4500 length 13
192.168.0.107:5353 : answer _services._dns-sd._udp.local. PTR _lxi._tcp.local. rclass 0x1 ttl 4500 length 7
192.168.0.107:5353 : answer _services._dns-sd._udp.local. PTR _scpi-raw._tcp.local. rclass 0x1 ttl 4500 length 12
192.168.0.107:5353 : answer _services._dns-sd._udp.local. PTR _http._tcp.local. rclass 0x1 ttl 4500 length 13
192.168.0.107:5353 : answer _services._dns-sd._udp.local. PTR _lxi._tcp.local. rclass 0x1 ttl 4500 length 7
192.168.0.107:5353 : answer _services._dns-sd._udp.local. PTR _scpi-raw._tcp.local. rclass 0x1 ttl 4500 length 12
192.168.0.107:5353 : answer _services._dns-sd._udp.local. PTR _http._tcp.local. rclass 0x1 ttl 4500 length 13
192.168.0.107:5353 : answer _services._dns-sd._udp.local. PTR _lxi._tcp.local. rclass 0x1 ttl 4500 length 7
192.168.0.107:5353 : answer _services._dns-sd._udp.local. PTR _scpi-raw._tcp.local. rclass 0x1 ttl 4500 length 12
192.168.0.125:5353 : answer _services._dns-sd._udp.local. PTR _smb._tcp.local. rclass 0x1 ttl 4500 length 12
192.168.0.125:5353 : answer _services._dns-sd._udp.local. PTR _device-info._tcp.local. rclass 0x1 ttl 4500 length 15
192.168.0.125:5353 : answer _services._dns-sd._udp.local. PTR _smb._tcp.local. rclass 0x1 ttl 4500 length 12
192.168.0.125:5353 : answer _services._dns-sd._udp.local. PTR _device-info._tcp.local. rclass 0x1 ttl 4500 length 15
192.168.0.125:5353 : answer _services._dns-sd._udp.local. PTR _smb._tcp.local. rclass 0x1 ttl 4500 length 12
192.168.0.125:5353 : answer _services._dns-sd._udp.local. PTR _device-info._tcp.local. rclass 0x1 ttl 4500 length 15
192.168.122.1:5353 : answer _services._dns-sd._udp.local. PTR _smb._tcp.local. rclass 0x1 ttl 4500 length 12
192.168.122.1:5353 : answer _services._dns-sd._udp.local. PTR _device-info._tcp.local. rclass 0x1 ttl 4500 length 15
192.168.122.1:5353 : answer _services._dns-sd._udp.local. PTR _smb._tcp.local. rclass 0x1 ttl 4500 length 12
192.168.122.1:5353 : answer _services._dns-sd._udp.local. PTR _device-info._tcp.local. rclass 0x1 ttl 4500 length 15
192.168.122.1:5353 : answer _services._dns-sd._udp.local. PTR _smb._tcp.local. rclass 0x1 ttl 4500 length 12
192.168.122.1:5353 : answer _services._dns-sd._udp.local. PTR _device-info._tcp.local. rclass 0x1 ttl 4500 length 15
10.65.19.1:5353 : answer _services._dns-sd._udp.local. PTR _smb._tcp.local. rclass 0x1 ttl 4500 length 12
10.65.19.1:5353 : answer _services._dns-sd._udp.local. PTR _device-info._tcp.local. rclass 0x1 ttl 4500 length 15
10.65.19.1:5353 : answer _services._dns-sd._udp.local. PTR _smb._tcp.local. rclass 0x1 ttl 4500 length 12
10.65.19.1:5353 : answer _services._dns-sd._udp.local. PTR _device-info._tcp.local. rclass 0x1 ttl 4500 length 15
10.65.19.1:5353 : answer _services._dns-sd._udp.local. PTR _smb._tcp.local. rclass 0x1 ttl 4500 length 12
10.65.19.1:5353 : answer _services._dns-sd._udp.local. PTR _device-info._tcp.local. rclass 0x1 ttl 4500 length 15
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _pdl-datastream._tcp.local. rclass 0x1 ttl 4500 length 23
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _printer._tcp.local. rclass 0x1 ttl 4500 length 11
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _ipp._tcp.local. rclass 0x1 ttl 4500 length 7
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _scanner._tcp.local. rclass 0x1 ttl 4500 length 11
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _http._tcp.local. rclass 0x1 ttl 4500 length 8
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _privet._tcp.local. rclass 0x1 ttl 4500 length 10
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _uscan._tcp.local. rclass 0x1 ttl 4500 length 9
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _pdl-datastream._tcp.local. rclass 0x1 ttl 4500 length 23
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _printer._tcp.local. rclass 0x1 ttl 4500 length 11
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _ipp._tcp.local. rclass 0x1 ttl 4500 length 7
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _scanner._tcp.local. rclass 0x1 ttl 4500 length 11
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _http._tcp.local. rclass 0x1 ttl 4500 length 8
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _privet._tcp.local. rclass 0x1 ttl 4500 length 10
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _uscan._tcp.local. rclass 0x1 ttl 4500 length 9
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _pdl-datastream._tcp.local. rclass 0x1 ttl 4500 length 23
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _printer._tcp.local. rclass 0x1 ttl 4500 length 11
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _ipp._tcp.local. rclass 0x1 ttl 4500 length 7
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _scanner._tcp.local. rclass 0x1 ttl 4500 length 11
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _http._tcp.local. rclass 0x1 ttl 4500 length 8
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _privet._tcp.local. rclass 0x1 ttl 4500 length 10
192.168.0.188:5353 : answer _services._dns-sd._udp.local. PTR _uscan._tcp.local. rclass 0x1 ttl 4500 length 9
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _http._tcp.local. rclass 0x1 ttl 4500 length 18
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _lxi._tcp.local. rclass 0x1 ttl 4500 length 17
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _vxi-11._tcp.local. rclass 0x1 ttl 4500 length 20
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _scpi-raw._tcp.local. rclass 0x1 ttl 4500 length 22
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _hislip._tcp.local. rclass 0x1 ttl 4500 length 20
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _scpi-telnet._tcp.local. rclass 0x1 ttl 4500 length 25
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _http._tcp.local. rclass 0x1 ttl 4500 length 18
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _lxi._tcp.local. rclass 0x1 ttl 4500 length 17
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _vxi-11._tcp.local. rclass 0x1 ttl 4500 length 20
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _scpi-raw._tcp.local. rclass 0x1 ttl 4500 length 22
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _hislip._tcp.local. rclass 0x1 ttl 4500 length 20
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _scpi-telnet._tcp.local. rclass 0x1 ttl 4500 length 25
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _http._tcp.local. rclass 0x1 ttl 4500 length 18
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _lxi._tcp.local. rclass 0x1 ttl 4500 length 17
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _vxi-11._tcp.local. rclass 0x1 ttl 4500 length 20
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _scpi-raw._tcp.local. rclass 0x1 ttl 4500 length 22
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _hislip._tcp.local. rclass 0x1 ttl 4500 length 20
192.168.0.117:5353 : answer _services._dns-sd._udp.local. PTR _scpi-telnet._tcp.local. rclass 0x1 ttl 4500 length 25
Closed sockets

lundmar@wopr:~/test/git/mdns/bin$ git diff
diff --git a/mdns.c b/mdns.c
index 433936d..cbc6f06 100644
--- a/mdns.c
+++ b/mdns.c
@@ -702,7 +702,7 @@ open_service_sockets(int* sockets, int max_sockets) {
 static int
 send_dns_sd(void) {
        int sockets[32];
-       int num_sockets = open_client_sockets(sockets, sizeof(sockets) / sizeof(sockets[0]), 0);
+       int num_sockets = open_client_sockets(sockets, sizeof(sockets) / sizeof(sockets[0]), MDNS_PORT);
        if (num_sockets <= 0) {
                printf("Failed to open any client sockets\n");
                return -1;
diff --git a/mdns.h b/mdns.h
index 4b3b03d..c0af3f1 100644
--- a/mdns.h
+++ b/mdns.h
@@ -868,7 +868,7 @@ static const uint8_t mdns_services_query[] = {
     // PTR record
     0x00, MDNS_RECORDTYPE_PTR,
     // QU (unicast response) and class IN
-    0x80, MDNS_CLASS_IN};
+    0x00, MDNS_CLASS_IN};

 static inline int
 mdns_discovery_send(int sock) {

[lundmar]

Here is the wireshark trace matching the example run: https://file.io/dxPKsS478MtD

[belotv]

Ok good, the new implementation should work - will send it to you for a test on Sunday. I forgot to mention that for the previous small test you should use the socket_listen callback rather than the discovery_recv to avoid the filtering on the services query as this device does not reply with “services”.

Kr

Envoyé de mon iPhone

Le 11 févr. 2022 à 11:57, Martin Lund @.***> a écrit :

 Here is the wireshark trace matching the example run: https://file.io/dxPKsS478MtD

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.

[lundmar]

Great. I'll test it before we merge.

[belotv]

Available here for testing: https://github.com/belotv/liblxi

Hopefully this version will handle all potential cases (still have some code cleanup to do).

[lundmar]

I just tested it and it only finds 1 instrument:

lundmar@wopr:~/opt$ LD_PRELOAD=$HOME/opt/liblxi/lib/x86_64-linux-gnu/liblxi.so ./lxi-tools/bin/lxi discover -m
Searching for LXI devices - please wait...

Broadcasting on interface enxe4b97a86fdad
Broadcasting on interface mpqemubr0
Broadcasting on interface virbr0
  Found "Rohde&Schwarz power supply NGM202-101403" on address 192.168.0.107
    lxi service on port 80
  Found "Rohde&Schwarz power supply NGM202-101403" on address 192.168.0.107
    scpi-raw service on port 5025

Found 2 services