IS-04: Automate remaining workshop/interop checklist items for Discovery & Registration

[garethsb]

Based on the most recent interop checklists, the following items remain to be automated if possible. NB: Not all test cases are applicable for all Device Classes.

AMWA NMOS Discovery & Registration (IS-04) Checklist

Version 2.2 (17th July 2018)

Section 1: Discovery & Registration

NB: Each of the below items should be checked off against each API version implemented.

• [x] 1.1 Node can discover network registration service via mDNS
• [x] 1.2 Node can discover network registration service via unicast DNS
• [ ] 1.3 Node re-queries unicast/multicast DNS when its Registration API connection fails and the DNS TTL expires
• [ ] 1.4 Node does not re-query unicast/multicast DNS for known records unless the TTL expires
• [ ] 1.5 Node does not register with registries set to DNS-SD TXT priority>=100 unless explicitly configured to
• [x] 1.6 Node registers with registries based upon their DNS-SD TXT priority order (lowest first)
• [x] 1.7 Node can register a valid Node resource with the network registration service, matching its Node API self resource
• [x] 1.8 Node maintains itself in the registry via periodic calls to the health resource
• [x] 1.9 Node correctly handles HTTP 4XX and 5XX codes from the registry, re-registering or trying alternative Registration APIs as required
• [x] 1.10 Node can register a valid Device resource with the network registration service, matching its Node API Device resource
• [x] 1.11 Node can register a valid Source resource with the network registration service, matching its Node API Source resource
• [x] 1.12 Node can register a valid Flow resource with the network registration service, matching its Node API Flow resource
• [x] 1.13 Node can register a valid Sender resource with the network registration service, matching its Node API Sender resource
• [x] 1.14 Node can register a valid Receiver resource with the network registration service, matching its Node API Receiver resource
• [x] 1.15 Node advertises a Node type mDNS announcement with no ver_* TXT records in the presence of a Registration API
• [x] 1.16 Node advertises a Node type mDNS announcement with ver_* TXT records in the absence of a Registration API
• [ ] 1.17 Node does not re-issue mDNS query answers periodically (ie. against DNS-SD spec)
• [ ] 1.18 **Node can discover other Nodes’ resources via mDNS, populating a local interface as specific by the P2P discovery scheme
• [x] 1.19 Node successfully re-registers with the Registration API if its network link is unplugged for 15 seconds (assuming default timeouts and 404 heartbeat response)
• [x] 1.20 Node does not re-register itself with the Registration API if its network link is unplugged for between 5 and 10 seconds and then re-plugged (assuming default timeouts and 200 heartbeat response)
• [x] 1.21 Node successfully discovers and heartbeats with a second registry if the first one goes offline

**Local registry population is only of use to devices which can display a list of sources to connect to via a user interface. This targets display and control devices (including camera viewfinders which can pick up PGM out).

[andrewbonney]

Another one for this category: Node correctly observes api_proto and api_ver TXT records. We may need to move these into a separate test set as it'll require a few more mDNS transactions and might slow the main IS0401 set down too much.

EDIT: The above is added by https://github.com/AMWA-TV/nmos-testing/pull/232

This should include testing of whether a Node correctly transitions between two API versions if it supports multiple at the same time.

[andrewbonney]

Another important enhancement in this area. The mock registry accepts any version of registration for ease in supporting multiple versions, but we don't currently check that this version matches the API version under test (and that of the mDNS/uDNS records).

EDIT: Added in https://github.com/AMWA-TV/nmos-testing/pull/233

[garethsb]

Are 1.19 and 1.20 covered by the connection-timeout test and other tests added to IS0401Test recently?

[andrewbonney]

As much as I think it will ever be possible to automatically test those, yes.