mobile-dev-inc / maestro

Painless Mobile UI Automation
https://maestro.mobile.dev/
Apache License 2.0
5.88k stars 281 forks source link

Assertions on HTML (webview) not working #2064

Open testcenter opened 2 months ago

testcenter commented 2 months ago

Is there an existing issue for this?

Steps to reproduce

The problem is related to maestro tests on Android. The Bitmovin player uses a Webview with HTML/Javascript to visualize the video player UI elements.

config.yaml

flows:
  - "DisplaysSubtitles.yaml"

DisplaysSubtitles.yaml

appId: com.bitmovin.player.samples.playback.basic
name: "Playback: Displays subtitles"
---
- launchApp
# toggle settings
- tapOn:
    text: "settings"
    retryTapIfNoChange: false
- extendedWaitUntil:
    visible: "subtitles"
    timeout: 10000
# open subtitle menu
- tapOn: "Select subtitle"
# select subtitle "en"
- tapOn: "en"
# !!! following assertion fails !!!
- assertVisible: "en"
# close settings menu
- tapOn:
    text: "settings"
    retryTapIfNoChange: false
# start playing
- tapOn:
    text: "Play/Pause"
    retryTapIfNoChange: false
# subtitle must be visible
- assertVisible: "Welcome to bitmovin's bitdash!"
- stopApp

Actual results

The assertion - assertVisible: "en" fails although visible. (inside a WebView) Using google_apis_playstore instead of google_apis would work

Expected results

Assertions behave the same on google_apis and google_apis_playstore (might be a general issue when parsing WebView hierarchy of google_apis vs google_apis_playstore)

About app

About environment

java -version

openjdk version "17.0.12" 2024-07-16
OpenJDK Runtime Environment (build 17.0.12+7-Ubuntu-1ubuntu222.04)
OpenJDK 64-Bit Server VM (build 17.0.12+7-Ubuntu-1ubuntu222.04, mixed mode, sharing)

uname -a

Linux ubuntu 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

please note, the problem also happens on maestro cloud!

Logs

Logs ``` Waiting for flows to complete... [Failed] Playback: Displays subtitles (35s) (Assertion is false: "en" is visible) 1/1 Flow Failed ```

Maestro version

1.38.1

How did you install Maestro?

install script (https://get.maestro.mobile.dev)

Anything else?

No response