IS-04: Automate remaining workshop/interop checklist items for Registries

[garethsb]

Based on the most recent interop checklists, the following items remain to be automated if possible. NB: Not all test cases are applicable for all Device Classes.

AMWA NMOS Discovery & Registration (IS-04) Checklist

Version 2.2 (17th July 2018)

Section 5: Registries

NB: Each of the below items should be checked off against each API version implemented.

• [x] 5.1 Registration API is advertised using mDNS, with appropriate TXT records
• [x] 5.2 Query API is advertised using mDNS, with appropriate TXT records
• [x] 5.3 Registration API accepts and stores a valid Node resource
• [x] 5.4 Registration API rejects an invalid Node resource with a 400 HTTP code
• [x] 5.5 Registration API accepts and stores a valid Device resource
• [x] 5.6 Registration API rejects an invalid Device resource with a 400 HTTP code
• [x] 5.7 Registration API accepts and stores a valid Source resource
• [x] 5.8 Registration API rejects an invalid Source resource with a 400 HTTP code
• [x] 5.9 Registration API accepts and stores a valid Flow resource
• [x] 5.10 Registration API rejects an invalid Flow resource with a 400 HTTP code
• [x] 5.11 Registration API accepts and stores a valid Sender resource
• [x] 5.12 Registration API rejects an invalid Sender resource with a 400 HTTP code
• [x] 5.13 Registration API accepts and stores a valid Receiver resource
• [x] 5.14 Registration API rejects an invalid Receiver resource with a 400 HTTP code
• [x] 5.15 Registration API accepts heartbeat requests for a Node held in the registry
• [x] 5.16 Registration API cleans up Nodes and their sub-resources when a heartbeat doesn’t occur for the duration of a fixed timeout period
• [x] 5.17 Query API supports browsing Node, Device, Source, Flow, Sender and Receiver resources via HTTP
• [x] 5.18 Query API supports persistent websocket connections to each resource type, created via the /subscriptions resource
• [x] 5.19 Query API supports basic query parameter filtering
• [x] 5.20 Query API supports advanced query parameter filtering (v1.1+)
• [ ] 5.21 Query API supports query parameters in websocket connections
• [ ] 5.22 Two instances of the same Registry Server are shown to operate in a cluster, sharing registered data
• [ ] 5.23 A Node can be shown to register with one clustered Registration API, before moving to another Registration API in the cluster without the Registry Server losing registered data

[garethsb]

W.r.t. 5.17 Query API supports browsing Node, Device, Source, Flow, Sender and Receiver resources via HTTP, tests of correct pagination is covered by #17. W.r.t. 5.18 Query API supports persistent websocket connections to each resource type, created via the /subscriptions resource, and 5.21 Query API supports query parameters in websocket connections, those are also part of #19. W.r.t. 5.20 Query API supports advanced query parameter filtering (v1.1+), see feature request for more extensive testing of RQL support, #18, and downgrade queries, #20.

[andrewbonney]

Additional: Registries should be tested to ensure they perform referential integrity checking correctly at each API version, likely by trying to register some resources in the right/wrong order: https://github.com/AMWA-TV/nmos-discovery-registration/blob/v1.2.x/docs/4.1.%20Behaviour%20-%20Registration.md#referential-integrity

EDIT: This was added as test_26

[jasperpeeters]

Note for myself, after the main changes have been merged into the test suite, the following points should be improved:

• schema validation for every response
• unify json sample data loading (as done in websocket event test)
• catch errors when accessing json (probably not needed if schema validation is reliable)