[QUESTION] Asking help for project config

[oguzhanozgur]

Describe the bug

I installed the Test Harness Tool using Raspberry Pi. By typing the IP address, I can access the Tool's interface. After clicking "Add Project", a JSON structure appears in front of me that I can edit as "Project Config". How should I fill this structure? Can you create or explain a sample JSON based on the 3 items below?

1. The test will be applied to a Linux TV.
2. TV will be the Controller.
3. Thread will not be used, only Wi-Fi.

Also, how will this TV to be tested be connected to the tool? So, how will the tool recognize TV? Will a physical connection be required? If so, what will this physical connection look like?

In a case we ran for trial purposes, there is a command like this: ./chip-tool thermostat setpoint-raise-lower 0 10 1 1 Where to write this command?

I ask because I cannot see/find the answers to these questions in the "user manual".

I thank you in advance for your answers.

Steps to reproduce the behavior

No response

Expected behavior

No response

Log files

No response

PICS file

No response

Screenshots

No response

Environment

No response

Additional Information

No response

[ccruzagralopes]

Hi @oguzhanozgur,

• The first 2 items you listed have no impact on the project config manual setting. The test cases that should be executed are selected based on the PICS values provided.

• Regarding item 3: You need to fill the "ssid" and "password" fields in the "wifi" section. You can delete the "thread" section from the default project config.

• how will the tool recognize TV?

  • The tool will recognize the device by the "discriminator". You need to set the appropriate values in the "dut_config" section.

• In a case we ran for trial purposes, there is a command like this: ./chip-tool thermostat setpoint-raise-lower 0 10 1 1 Where to write this command?

  • I'm sorry, but I didn't understand this part. Why do you need to run this command on Test Harness?

[oguzhanozgur]

Thank you for your answer.

The tool will recognize the device by the "discriminator". You need to set the appropriate values in the "dut_config" section.

According to your description, Project Config will be like this:

{ "network": { "wifi":{ "ssid": "ourWifi", "password": "ourPassword" } }, "dut_config": { "discriminator": "3840", "setup_code": "20202021", "pairing_mode": "onnetwork", "chip_tool_timeout": null, "chip_tool_use_paa_certs": false }, "test_parameters": null }

But we still have confusion in the following parts:

1. How will we fill in the values of the parameters in the dut_config section? Where can we learn these values? (What I wanted to ask is, you said the connection will be made according to the discriminator. But how do we find out what this value is? Of course, not only this; what values will the setup_code, pairing_mode, chip_tool_timeout and chip_tool_use_paa_certs parameters have and where can we learn these values?)
2. Also, how do we fill in the test_parameters parameter located just below? Or should we remove it from this structure?

I'm sorry, but I didn't understand this part. Why do you need to run this command on Test Harness?

We encountered this command as you can see from the attached image. How do we proceed here?

Also, I am getting error like this.

I am grateful for your answers in advance.

[ccruzagralopes]

How will we fill in the values of the parameters in the dut_config section?

These values are determined by the device, I can't say how to get them for your TV. The values that are filled by default are the ones used by the example apps provided by the Matter SDK, which are available in the apps folder. pairing_mode must be one of the following: onnetwork (if the DUT is already present on the operational network), ble-wifi or ble-thread. It will depend on the type of connection the DUT supports. If the DUT requires a PAA certificate to perform a pairing operation, chip_tool_use_paa_certs should be set to true.

Also, how do we fill in the test_parameters parameter located just below? Or should we remove it from this structure?

test_parameters are used for additional parameters that some test cases can have. For example: in the image below a test case YAML file defines some parameters with default values (under config). If your DUT needs other values for these parameters, you can override the values by adding those parameters in the test_parameters section of the Project Config. If you don't need to set any additional parameters, you can remove this key from the Project Config.

[ccruzagralopes]

We encountered this command as you can see from the attached image. How do we proceed here?

Manual steps are supposed to be performed outside of TH. You can use the terminal connected to the raspberry pi to run the command. The chip-tool binary is located in the apps folder. You can navigate to that folder and perform the commands there. There are instructions on the user guide for this:

• When running a semi-automated test case
• When running a manual test case

[ccruzagralopes]

Regarding the error: please run the following command and share the output: docker logs certification-tool-backend-1

[oguzhanozgur]

Regarding the error: please run the following command and share the output: docker logs certification-tool-backend-1

Where to run this command?

Thanks for other replies. Since we got an error with the updated project Config, I will look at the other results after solving this problem.

[ccruzagralopes]

Where to run this command?

On the terminal, in the host (raspberry pi)

[oguzhanozgur]

Where to run this command?

On the terminal, in the host (raspberry pi) I'm getting error when I try to run this command.

[ccruzagralopes]

Please run docker ps to list the running containers

[oguzhanozgur]

Thanks for your quick feedback. Here is the result of "docker ps":

[ccruzagralopes]

Ok. You can run docker logs certification-tool_backend_1 to get the logs

[oguzhanozgur]

log.txt

[ccruzagralopes]

Please restart the containers by running the following commands and verify if the error persists:

# in certification-tool repository:
./scripts/stop.sh
./scripts/start.sh

Wait a few minutes for the application to start.

[oguzhanozgur]

I restarted the containers by running the following commands. Verified : Error persists.

[ccruzagralopes]

Please send me the backend container logs again and also the output of this command: ./scripts/th/th-doctor.sh --complete

[oguzhanozgur]

Here is the output of the command: doctor.txt

And here is the backend container logs: log2.txt

(Since I used the tool a little during the day, the logs may be different from before.)

Thanks.

[oguzhanozgur]

When I try to change Project Config (by removing thread parameter), only these logs are added as new.

newLog.txt

Maybe it will be useful to you.

[ccruzagralopes]

Thanks for the files. It seems that you're running an old version of the tool. Please perform the following steps to update it:

cd certification-tool
./scripts/stop.sh 
cd ..
sudo rm -rf certification-tool/

git clone https://github.com/project-chip/certification-tool.git
cd certification-tool
git fetch
git checkout v2.10-beta2.2+spring2024
git submodule update --init --recursive
./scripts/ubuntu/auto-update.sh v2.10-beta2.2+spring2024
./scripts/start.sh

[ccruzagralopes]

Also, I'm sorry, but I realized that I gave you an incorrect information before. The test_parameters key in the Project Config will always be there, you don't need to remove it, just keep it as null if you don't need to use it.

[oguzhanozgur]

Hi,

I performed the steps but when I try to enter tool : there is an error.

Version: 2.10-beta2.2+spring2024 Sha: 2432a29

Also, to summarize this Project Config structure, can you write an example JSON for our scenario?

Thanks.

[ccruzagralopes]

I'm sorry, but I can't see what could be causing this error, so I'll suggest that you flash the latest image to get a clean environment. You can get the latest SD card image in this link: https://drive.google.com/drive/folders/1rBKbS5yp8jUnoeJO9gCv9sAxmI8dFw7L?usp=share_link

Regarding the Project Config structure, the example you gave in this message is ok.

[oguzhanozgur]

So, how should the Project Config be?

[ccruzagralopes]

{ "network": { "wifi":{ "ssid": "yourWifi", "password": "yourPassword" } }, "dut_config": { "discriminator": "yourDiscriminator", "setup_code": "yourSetupCode", "pairing_mode": "onnetwork", "chip_tool_timeout": null, "chip_tool_use_paa_certs": false }, "test_parameters": null }

[oguzhanozgur]

Hi again,

Unfortunately i'm facing same error. Version: 2.10-beta2.2+spring2024

[ccruzagralopes]

That's odd. @oguzhanozgur can you please detail the steps you're taking and what happens right before the error? Is it something that happens every time after a certain action? Or does the error appear "out of nowhere"? From the logs that you sent it seems that you were able to start a test execution at least one before the error.

[oguzhanozgur]

slackLog.txt

[oguzhanozgur]

That's odd. @oguzhanozgur can you please detail the steps you're taking and what happens right before the error? Is it something that happens every time after a certain action? Or does the error appear "out of nowhere"? From the logs that you sent it seems that you were able to start a test execution at least one before the error.

Hi again,

I enter the IP address of the Raspberry Pi from the browser and open the Tool. Afterwards, I try to create a new project. I encounter this problem when I edit the Project Config section (write the JSON you provided) and update it and click on create the project. So in summary, I face the same problem every time I try. However, when I try to create a project with the default value without making any changes in Project Config, I can create it successfully.

[fabiowmm]

@oguzhanozgur we can't identify anything wrong with your setup that would justify you not being able to connect to backend after changing the project config file. We had a couple situations like that in the past and it ended up being hardware issues. Do you have other raspi and sd card available by any chance?

[gladystonfranca]

@oguzhanozgur there is a problem with the example as thread section is currently mandatory on the schema.

So, just keep the thread entry as is (from the template) it will be ignored depending on the pairing_mode option.

An issue will be opened to review dependencies on the schema.

Please confirm if keeping the thread entry solves your problem.

[oguzhanozgur]

When I kept thread section, there is no problem. So, we can close this issue