[Feature] TH should name all logs properly according to TEDS alias to avoid manual renaming

project-chip / certification-tool

A test harness and tooling designed to simplify development, testing, and certification for devices, guided by the Connectivity Standards Alliance.

https://csa-iot.org/

Apache License 2.0

42 stars 24 forks source link

[Feature] TH should name all logs properly according to TEDS alias to avoid manual renaming #370

Open tcarmelveilleux opened 3 months ago

tcarmelveilleux commented 3 months ago

Feature description

All log files that can come out of TH (and Python tests) should be auto-named to have the TEDS alias so that renaming of test logs is not required.

Use Cases

Test events require TEDS aliases to be present in filenames
Doing these renames is very time consuming.

Test Cases

No response

Additional Information

No response

sander-gitl commented 3 months ago

As discussed, while some automation is possible: TEDS is a separate tool that will not be integrated (by design) into the TH tooling. As such, the best way is to think about means to simplify entry by providing e.g. a script that does this naming based on a one-time configuration by the user (to set the correct DUT alias, e.g.) and then uses that to provide the correct log file name: semi-automated.

tcarmelveilleux commented 3 months ago

Whenever I submit a log:

TEDS knows the alias
TEDS knows the test case ID (I just had to select it)
TEDS knows the type of log (I had to select it).

Given that, I should be able to upload any filename, and on submitting the TEDS result, the name should be renamed internally by TEDS before the data is written where it goes. This would remove all need to rename any files.

rquidute commented 2 months ago

Just for the record, currently TH generates the execution log file name following this template: ”{test_run_execution.id}-{test_run_execution.title}.log" For example: UI_Test_Run_2024_08_21_18_53_57.log

sander-gitl commented 2 months ago

For now, consider the TEDS part fixed by means of providing the corrected filenames to ATLs (the main recipient for the files) in the form ([TCID]_[DUTALIAS]originalfilename.originalextension). E.g. "[TC-DD-2.1][qU2d24m]_UI_Test_Run_2024_08_21_18_53_57.log". This will retain the original filename btw, but members no longer need to do an additional action here.

rquidute commented 2 months ago

Hi @sander-gitl and @tcarmelveilleux I'm not sure if I quite understand the goal of this issue. At first, it seems to me that it sounds like a new feature for TEADs (According to this note: Given that, I should be able to upload any filename...) but also it seems to TH should change the way it generates the execution log file name by using this template: [TCID]_[DUTALIAS]originalfilename.originalextension.

Currently TH generates only 1 file output which contains all test cases in the test run execution and according to the given template, the Test case name (TCID) should be in the file name.

Could you please clarify to me?