AMWA-TV / nmos-testing

Testing tool for the AMWA NMOS Specifications
https://specs.amwa.tv/nmos-testing/
Apache License 2.0
41 stars 48 forks source link

[IS-05] test_25 doesn't check for requested_time in /active endpoint #840

Open matthewdicken opened 10 months ago

matthewdicken commented 10 months ago

The test passes regardless of whether /active returns a value on

vis-a-vis Slack / AMWA Workshops #general:

Matt Dicken (Open Broadcast Systems) https://specs.amwa.tv/is-05/releases/v1.1.2/examples/sender-active-get.html Example of sender-active-get: "activation": { "mode": "activate_immediate", "requested_time": "1496759200:0", "activation_time": "1496759200:0"

MD: the description in the schema doesnt actually talk about the /active endpoint, so is the example wrong, or does the test suite not do the right thing?

Gareth S-B (NVIDIA)

the description in the schema doesnt actually talk about the /active endpoint I believe the intention is that the initial sentence of each attribute description in the schema applies to all responses on both /active and /staged, and then exceptions are made e.g. for PATCH /staged response and GET /staged after an immediate activation. In other words, the /active endpoint after any activation should have all non-null values for mode, requested_time and activation_time.

MD: So just to confirm: you would suggest that the requested_time should be populated on /active, even if the test isn't specifically looking for that?

GSB: Yes

GSB: The testing tool doesn't check for a specific value of requested_time because it can't put very tight bounds on the value... I guess it could check it was non-null, and <= activation_time

garethsb commented 10 months ago

Another thing that became clear during Slack chat that @matthewdicken quoted, was that some of the error messages, like "Activation entries were not set at {} after {} activation" cover a few different problems with different properties across different endpoint/verb responses. For example, after an immediate activation, activation_time in the response to GET /active should be equal to activation_time in the response to PATCH /staged (the test allows >=).