project-chip / matter-test-scripts

Test scripts related to Matter Certification
https://csa-iot.org/
Apache License 2.0
9 stars 3 forks source link

Python testing: change input to self.wait_for_user_input to work with TH #243

Closed cecille closed 2 months ago

cecille commented 5 months ago

Tests requiring fixes:

cecille commented 4 months ago

@rquidute - would you mind clarifying where the input message and prompt string should show up? Is one only for the TH?

rquidute commented 4 months ago

Hi @cecille, this PR (https://github.com/project-chip/connectedhomeip/pull/32299 )has an example on how to use the self.wait_for_user_input call.

The method documentation at matter_testing_support.py file has the following documentation:

"""Ask for user input and wait for it.

    Args:
        prompt_msg (str): Message for TH UI prompt. Indicates what is expected from the user.
        input_msg (str, optional): Prompt for input function, used when running tests manually. Defaults to "Press Enter when done.\n".
        prompt_msg_placeholder (str, optional): TH UI prompt input placeholder. Defaults to "Submit anything to continue".
        default_value (str, optional): TH UI prompt default value. Defaults to "y".

    Returns:
        str: User input

@cecille please, let me know if more clarification is necessary.

cecille commented 4 months ago

couple questions 1) Why is the input_msg different than the prompt message? 2) What is the difference between the prompt_message and the prompt_msg_placeholder? one says 'Message for TH UI prompt.", other says "TH UI prompt input placeholder". Is this in relation to a GUI? ie, is the placeholder text what appears where the user is supposed to type? if that's in the input box, then what's the default value?

rquidute commented 4 months ago

Hi @cecille, providing more details as follow:

Why is the input_msg different than the prompt message?

The input_msg parameter will not be sent to TH. The prompt_message is the text that will be displayed as a prompt message to the user.

What is the difference between the prompt_message and the prompt_msg_placeholder? one says 'Message for TH UI prompt.", other says "TH UI prompt input placeholder". Is this in relation to a GUI? ie, is the placeholder text what appears where the user is supposed to type? if that's in the input box, then what's the default value?

The prompt will be displayed as in the screenshot bellow where prompt_msg_placeholder is the prompt place holder with a grayed text = Submit anything to continue and the prompt_message is the message displayed in the prompt in the example with the text Reset DUT

Screenshot 2024-05-15 at 10 54 00

According to this PR https://github.com/project-chip/connectedhomeip/pull/32299 , we are considering just passing the prompt_message in self.wait_for_user_input call, as the other parameters we considered using the defaults.

In addition, the default_value parameter, is the expected user response to the input (input method that was moved from test case to the self.wait_for_user_input method).

Screenshot 2024-05-15 at 11 06 16
austina-csa commented 4 months ago

I have a few questions about this:

rquidute commented 4 months ago

Hi @austina-csa, I'm sharing some feedback:

Are these tests specifically meant to be run in the GUI? So far, I have only run them from the CLI via the command: ./scripts/tests/run_python_test.py

Inside self.wait_for_user_input() method, there is a check for the runner_hook, this is only used by TH UI, so in case you are not running from TH UI, it should behave same way as just having the input("Input message")

In many of these scripts, before the self.wait_for_user_input line is executed, the self.print_step line is executed, explaining the step. Wouldn't this be redundant to pass step_name afterwards?

cecille commented 4 months ago

@austina-csa - API suggestion - let's remove the input_msg from the API, and use the prompt_msg for the input call (ie, change the function in matter_testing_support.py to be

    def wait_for_user_input(self,
                            prompt_msg: str,
                            prompt_msg_placeholder: str = "Submit anything to continue",
                            default_value: str = "y") -> str:
        """Ask for user input and wait for it.

        Args:
            prompt_msg (str): Message for TH UI prompt and input function. Indicates what is expected from the user.
          prompt_msg_placeholder (str, optional): TH UI prompt input placeholder (where the user types). Defaults to "Submit anything to continue".
            default_value (str, optional): TH UI prompt default value. Defaults to "y".

        Returns:
            str: User input
        """
        if self.runner_hook:
            self.runner_hook.show_prompt(msg=prompt_msg,
                                         placeholder=prompt_msg_placeholder,
                                         default_value=default_value)
        return input(prompt_msg)

Then just do a straight-up search and replace for input with wait_for_user_input

austina-csa commented 4 months ago

Hello Cecille, thanks for letting me know. I can make the necessary adjustments to the scripts that I have worked on.

cecille commented 2 months ago

https://github.com/project-chip/connectedhomeip/pull/33672