search

project-chip / certification-tool

A test harness and tooling designed to simplify development, testing, and certification for devices, guided by the Connectivity Standards Alliance.
https://csa-iot.org/
Apache License 2.0
42 stars 24 forks source link

[Feature] Indicate Endpoint under test in UI as a test runs on a given endpoint #367

Open tcarmelveilleux opened 3 months ago

tcarmelveilleux commented 3 months ago

Feature description

Every time a test runs on an endpoint, I can't tell which endpoint is targeted in the UI. I have to remember the endpoint test_parameter set in project and if it's forgotten to update test could run on wrong endpoint.

The TH UI should always include the current endpoint target by a test (if test is applicable to an endpoint and not global).

Use Cases

Running tests on devices with multiple endpoints (e.g. 0 and 1), which is all devices.

Test Cases

No response

Additional Information

No response

raul-marquez-csa commented 3 months ago

"I have to remember the endpoint test_parameter set in project": In the project config screen? "and if it's forgotten to update test could run on wrong endpoint": Again, in the project config screen?

"The TH UI should always include the current endpoint target by a test": Then is it per project or per test?

"(if test is applicable to an endpoint and not global)": Can you give more detail on this? if global, should it also be evident in the UI?

fabiowmm commented 3 months ago

@raju-apple @antonio-amjr I have confirmed the frontend has no means to always know which endpoint is being tested at a given point in time without support from backend/SDK. There exist dynamic endpoint tests for which the endpoint is not known in advance (see for instance https://github.com/project-chip/connectedhomeip/pull/34819#pullrequestreview-2272466759).

@cecille correct me if I got it wrong.

antonio-amjr commented 2 months ago

Hi everyone,

I'm proposing some changes on both SDK and Backend repos for this feature. Please take a look and feel free to comment and suggest:

@raul-marquez-csa Please take a special look to those changes if possible. If you spot any necessary change, specially if it's necessary to the frontend, let me know. The PRs above have images from tests I made of Frontend retrieving the data from the web socket and showing poorly just a for testing purpose. I not sure how the Endpoint should be presented on the UI.

@tcarmelveilleux Let me know if my take on those PRs are on track for what you requested here.

CC: @fabiowmm, @raju-apple, @cecille