ScreenPyHQ / screenpy

Screenplay pattern base for Python automated UI test suites.
MIT License
27 stars 3 forks source link

conditional actions #95

Closed MarcelWilson closed 9 months ago

MarcelWilson commented 1 year ago

I've been looking at the need for conditionals that maintain a screenplay pattern. One need that arises is the ability to perform See() without raising an exception.

actor.will(Conditionally(DoThing()).if_(bool_condition))

Invariably the condition is the result of a test that See handles. This got me thinking all we need to reuse See actions as a bool would be to catch the AssertionError. Something like this:

def is_successful(actor: Actor, action: Performable) -> bool:
    try:
        action.perform_as(actor)
    except AssertionError:
        return False
    return True

Logically good, but using the function isn't screenplay pattern friendly.

if is_successful(actor, See(question, resolution)):
    actor.will(DoThing())

Not to mention that the conditional would be pretty much limited to See only. Kinda useful, but I think we can do better...

Instead of passing in See a tester could pass in ANY action they wish. These actions could have any type of exception which would they consider as a 'failure' state. This means our above function needs to be customizable to any combination of exceptions. Ok, let's just add an argument to the function and let them pass in the exceptions they wish.

def is_successful(actor: Actor, action: Performable, failure_exception: tuple[type[BaseException]]) -> bool:
    try:
        action.perform_as(actor)
    except failure_exception:
        return False
    return True

So we can.... sigh... use it like... this.

if is_successful(actor, See(question, resolution), (AssertionError, CustomException)):
    actor.will(DoThing())

"ew david!" That's just gross. If only the action could tell us what exceptions are considered a failure. wait... they totally could.

A new Protocol could identify actions that contain the exception classes that, when caught, translate to a success boolean.

@runtime_checkable
class Failable(Performable, Protocol):
    failure_exceptions: tuple[type[BaseException]]
def is_successful(actor: Actor, action: Failable) -> bool:
    try:
        action.perform_as(actor)
    except action.failure_exceptions:
        return False
    return True

Ok, that's neat, but we still have this function that doesn't fit with the screenplay pattern. The way I see it the function probably needs to belong to the actor or the action.


Belong to the action:

The protocol could include the method and be inherited by the action

@runtime_checkable
class Failable(Performable, Protocol):
    failure_exceptions: tuple[type[BaseException]]

    def is_successful(self, actor: Actor) -> bool:
        try:
            self.perform_as(actor)
        except self.failure_exceptions:
            return False
        return True
class MyAction(Failable):
    def perform_as(self, the_actor: "Actor") -> None:
        ...
    failure_exceptions = (AssertionError, CustomException)
if MyAction().is_successful(actor):
    actor.will(DoThing)

Not terrible. It still doesn't really fit with the screenplay pattern because all the patterns are actor centric. In our above example the actor is passed in afterward when typically it is the actor doing the the action. In this case the conditional we're trying to conduct isn't an Performable. It can't be an action because we need the return from the function while Performable will always return None.

It sounds like the actor needs a new method that returns the result of our conditional check.


Belong to actor:

class Actor
    def is_successful(self, action: Failable) -> bool:
        try:
            action.perform_as(self)
        except action.failure_exceptions:
            return False
        return True
if actor.is_successful(MyAction()):
    actor.will(DoThing())

the names of these methods totally need some natural language consideration

MarcelWilson commented 1 year ago

"But Marcel, where does that leave the action Conditionally?"

It looks something like this...


def successful(actor: Actor, action: Failable) -> bool:
    try:
        action.perform_as(actor)
    except action.failure_exceptions:
        return False
    return True

class Conditionally(Performable):
    if_action: Failable
    then_action: Performable
    else_action: Performable | None = None
    if_action_to_log: str = ""
    then_action_to_log: str = ""
    else_action_to_log: str = ""
    # failure_exceptions = (AssertionError,)

    @beat("{} tries to Conditionally '{then_action_to_log}' if '{if_action_to_log}' is True {else_action_to_log}")
    def perform_as(self, actor: Actor):
        if successful(actor, self.if_action):
            actor.will(self.then_action)

        # #TODO: method is part of the action
        # if self.if_action.successful(actor):
        #     actor.will(self.then_action)

        # #TODO: method is part of the actor
        # if actor.successful(self.if_action):
        #     actor.will(self.then_action)

        else:
            if self.else_action:
                actor.will(self.else_action)

    def if_(self, if_action: Failable) -> Self:
        self.if_action = if_action
        self.if_action_to_log = get_additive_description(self.if_action)
        return self

    def else_(self, else_action3: Performable) -> Self:
        self.else_action = else_action3
        self.else_action_to_log = f"else '{get_additive_description(self.else_action)}'"
        return self

    def __init__(self, then_action: Performable):
        self.then_action = then_action
        self.then_action_to_log = get_additive_description(self.then_action)
actor.will(Conditionally(DoThing()).if_(InitialState()).else_(DoOther()))
MarcelWilson commented 1 year ago

I did not consider how logging would look when using successful(actor, See()) The describe is f"See if {self.question_to_log} is {self.resolution_to_log}." Which unfortunately doesn't lend itself to putting the statement in a @beat above the Conditionally.perform_as

    @beat("{} tries to Conditionally {then_action_to_log} if {if_action_to_log}")
    def perform_as(self, actor: Actor):
        ...
Marcel tries to Conditionally go to next page if see if the next_page_button is clickable

Not to mention what logs when the condition is false:

Marcel tries to Conditionally go to next page if see if the next_page_button is clickable 
    Marcel sees if the next_page_button is clickable.
        Marcel inspects the next_page_button.
            => <selenium.webdriver.remote.webelement.WebElement (session="851a8233d25ea93bb8d1963def911edd", element="FE258B1A836F1909CD409F0915EA4D57_element_61")>
        ... hoping it's clickable
            => False
        ***ERROR***

AssertionError: None
Expected: the element is enabled/clickable
     but: was not enabled/clickable
bandophahita commented 1 year ago

I forgot that we can use Silently to turn off the AssertionError portion of the log.

def successful(actor: Actor, action: Failable) -> bool:
    try:
        actor.perform(Silently(action))  # <-- wrap it here
    except action.failure_exceptions:
        return False
    return True
Marcel tries to Conditionally 'go to next page' if 'see if the next_page_button is clickable' is True 
    Marcel sees if the next_page_button is clickable.
        Marcel inspects the next_page_button.
            => <selenium.webdriver.remote.webelement.WebElement (session="2ad1d323a1355a46c995b1148f4bea0f", element="B0C269F336537B6111C02825CFF53607_element_71")>
        ... hoping it's clickable
            => False

That's a little better... but not great.

bandophahita commented 1 year ago

I feel I need to take a step back and consider possible scenarios


Situation - click button if it is clickable

The boolean condition is "button is clickable". At the core selenium level we need the following logic:

# first we need to find the element. (assuming it's even there)
element = driver.find_element(locator)
# next we check two different methods on the element in order to know if it's "clickable"
stmt = (element.is_displayed() and element.is_enabled())

We have this really nice test that does this but it raises an AssertionError when it fails.

actor.shall(See(Target("button").located_by(locator), IsClickable()))

The problem with trying to utilize See for a boolean is we dont have a good way to get the information out of See without catching the exception.

At the heart of See we call hamcrest's assert_that which goes through the process of comparing the question with the resolution. I keep thinking we need something similar to See that simply doesn't raise an assertion, it just returns the bool. But that might only be true for when we are using questions & resolutions.

In other words, we need a way to form bool stmts using screenplay pattern.(?)

Question: Are there other boolean stmts that don't consist of a question & resolution?

perrygoy commented 11 months ago

I just realized i never commented on this issue, we just talked about it on Discord and Slack.

I think the biggest part of our discussion was that the new Either Action in your PR #90 can be used for this purpose by using See. I think you even leaned into that by having Either only catch AssertionError by default.

bandophahita commented 11 months ago

A tester could very well use Either in this manner. However, they are limited to this:

if do x() fails:
    do y()

It doesn't give them the ability to conditionally do something else instead.

if do x() fails:
    do y()
else:
    do z()
perrygoy commented 11 months ago

I think you'll find you can do "else" in there, because your first case:

if do_x() fails:

actually has the condition set in it, because do_x() would really be a See() combined with another action.

if See():
    do_rest_of_x()
else:
    do_y()

But we can also do nested stuff.

the_actor.attempts_to(
    Either(
        Either(
            do_x()
        ).or_(
            do_y()
        ).ignoring()
    ).or_(
        Either(
            do_z()
        ).or_(
            do_something_else()
        )
    )
)
bandophahita commented 11 months ago

The problem with any of the actions being used in this manner is they don't return anything. All Performable return None when called.

Maybe I'm not seeing it, but I don't think you can use the logic from Either to accomplish the else clause.

perrygoy commented 11 months ago

Either doesn't depend on return values, it depends on exceptions being raised. If See() raises an AssertionError, then Either executes the "else" clause.

bandophahita commented 11 months ago

If it doesn't raise the exception?
That's the case try/except can't handle, but if/else does.

perrygoy commented 11 months ago

If it doesn't raise the exception?

No, that's the first bit:

if See():
    # no exception raised
    do_x()
else:
    # AssertionError was raised by See
    do_y()
bandophahita commented 11 months ago

Either can't execute the do_x. It executes the See and if that raises exception it executes do_y. If See doesn't raise the exception it moves on to next step.

perrygoy commented 11 months ago

Yes, that's what i'm saying.

the_actor.attempts_to(
    Either(
        See(...),
        DoX(),
    ).or_(
        DoY()
   )
)

This exactly replicates if/else behavior, with the See() as the condition.

if See(...):
    DoX()
else:
    DoY()
bandophahita commented 11 months ago

~Maybe I'm missing it.. can you write out an example of a true if/then/else case using Either to show me?~

Oh, I see what you're saying. You're example is backwards, but I think I get it.

the_actor.attempts_to(
    Either(
        See(...),    # this needs to pass in order to do the next step. 
        DoY(),
    ).or_(  # when See fails, the exception is caught and we move into executing the or steps
        DoX()
   )
)
bandophahita commented 11 months ago

While users could create the logic this way, reading the logging will not be as clear.

bandophahita commented 9 months ago

90 was merged