wix / Detox

Gray box end-to-end testing and automation framework for mobile apps
https://wix.github.io/Detox/
MIT License
11.04k stars 1.91k forks source link

Both Android and iOS e2e tests fail when following contribution guide #369

Closed SMJ93 closed 6 years ago

SMJ93 commented 6 years ago

Problem

I have followed the contribution guide.

lerna run test runs fine with all tests passing at 100%.

Then when I try to run detox e2e covarage tests on both Android and iOS they fail.

At this stage when I run git status I noticed detox/src/ios/earlgreyapi/GREYMatchers.js has been modified and several package-lock.json's installed (should be ignored / commited?).

iOS

cd detox/test
npm run build:ios
npm run e2e:ios

Full error:

  Sanity
    ✓ should have welcome screen (361ms)
    ✓ should show hello screen after tap (746ms)
    ✓ should show world screen after tap (243ms)

  Matchers
    ✓ should match elements by (accesibility) label (753ms)
    ✓ should match elements by (accesibility) id (290ms)
    ✓ should match elements by index (267ms)
    ✓ should match elements by type (native class) (278ms)
    ✓ :ios: should match elements by accesibility trait (268ms)
    ✓ should match elements with ancenstor (parent)
    ✓ should match elements with descendant (child)
    ✓ should match elements by using two matchers together with and
    - should choose from multiple elements matching the same matcher using index

  Actions
    ✓ should tap on an element (1163ms)
    ✓ should long press on an element (1920ms)
    ✓ should multi tap on an element (1554ms)
    ✓ should tap on an element at point (1484ms)
    1) should type in an element

  15 passing (23s)
  1 pending
  1 failing

  1) Actions should type in an element:
     Error: Error: An action failed. Please refer to the error trace below.
Exception with Action: {
  "Action Name" : "Type 'passcode'",
  "Element Matcher" : "(((respondsToSelector(accessibilityIdentifier) && accessibilityID('UniqueId937')) && !(kindOfClass('RCTScrollView'))) || (kindOfClass('UIScrollView') && ((kindOfClass('UIView') || respondsToSelector(accessibilityContainer)) && ancestorThatMatches(((respondsToSelector(accessibilityIdentifier) && accessibilityID('UniqueId937')) && kindOfClass('RCTScrollView'))))))"
}

Error Trace: [
  {
    "Description" : "Failed to type string 'passcode', because keyboard was not shown on screen.",
    "Error Domain" : "com.google.earlgrey.ElementInteractionErrorDomain",
    "Error Code" : "2",
    "File Name" : "GREYKeyboard.m",
    "Function Name" : "+[GREYKeyboard typeString:inFirstResponder:error:]",
    "Line" : "185"
  }
]

Android

cd detox/test
npm run build:android
npm run e2e:android

Full Android error:

detox verb ws
    1) :ios: should match elements by accesibility trait

detox verb ws send: {"type":"cleanup","params":{"stopRunner":false},"messageId":26}
detox verb ws onMessage: {"messageId":26,"type":"cleanupDone","params":{}}
detox verb ws

  7 passing (13s)
  1 failing

  1) Matchers :ios: should match elements by accesibility trait:
     Error: Error: 'is assignable from class: class android.view.View' matches multiple views in the hierarchy.
Problem views are marked with '****MATCHES****' below.

View Hierarchy:
+>DecorView{id=-1, visibility=VISIBLE, width=1080, height=1920, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=WM.LayoutParams{(0,0)(fillxfill) sim=#10 ty=1 fl=#81810100 pfl=0x20000 wanim=0x10302f6 needsMenuKey=2 colorMode=0}, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=3} ****MATCHES****
|
+->LinearLayout{id=-1, visibility=VISIBLE, width=1080, height=1794, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@3aa68fe, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=2} ****MATCHES****
|
+-->ViewStub{id=16908678, res-name=action_mode_bar_stub, visibility=GONE, width=0, height=0, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=true, is-selected=false, layout-params=android.widget.LinearLayout$LayoutParams@e00485f, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0} ****MATCHES****
|
+-->FrameLayout{id=16908290, res-name=content, visibility=VISIBLE, width=1080, height=1731, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.widget.LinearLayout$LayoutParams@7a0eb75, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=63.0, child-count=1} ****MATCHES****
|
+--->ReactRootView{id=81, visibility=VISIBLE, width=1080, height=1731, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@beec629, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, child-count=12} ****MATCHES****
|
+---->ReactViewGroup{id=53, visibility=VISIBLE, width=90, height=103, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=true, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@62625f9, tag=null, root-is-layout-requested=false, has-input-connection=false, x=495.0, y=341.0, child-count=1} ****MATCHES****
|
+----->ReactTextView{id=52, desc=Label, visibility=VISIBLE, width=90, height=51, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.view.ViewGroup$LayoutParams@566393e, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, text=Label, input-type=0, ime-target=false, has-links=false} ****MATCHES****
|
+---->ReactViewGroup{id=56, visibility=VISIBLE, width=34, height=104, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=true, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@5eb739f, tag=null, root-is-layout-requested=false, has-input-connection=false, x=523.0, y=444.0, child-count=1} ****MATCHES****
|
+----->ReactTextView{id=55, visibility=VISIBLE, width=34, height=51, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.view.ViewGroup$LayoutParams@e74dcec, tag=UniqueId345, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, text=ID, input-type=0, ime-target=false, has-links=false} ****MATCHES****
|
+---->ReactViewGroup{id=59, visibility=VISIBLE, width=93, height=77, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=true, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@d9a28b5, tag=null, root-is-layout-requested=false, has-input-connection=false, x=494.0, y=548.0, child-count=1} ****MATCHES****
|
+----->ReactTextView{id=58, visibility=VISIBLE, width=93, height=51, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.view.ViewGroup$LayoutParams@408ab4a, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, text=Traits, input-type=0, ime-target=false, has-links=false} ****MATCHES****
|
+---->ReactViewGroup{id=62, visibility=VISIBLE, width=132, height=157, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=true, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@11daabb, tag=null, root-is-layout-requested=false, has-input-connection=false, x=474.0, y=625.0, child-count=1} ****MATCHES****
|
+----->ReactImageView{id=60, visibility=VISIBLE, width=132, height=131, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.view.ViewGroup$LayoutParams@b48dbd8, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0} ****MATCHES****
|
+---->ReactViewGroup{id=66, visibility=VISIBLE, width=168, height=168, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@2119b31, tag=Grandfather883, root-is-layout-requested=false, has-input-connection=false, x=456.0, y=782.0, child-count=1} ****MATCHES****
|
+----->ReactViewGroup{id=65, visibility=VISIBLE, width=126, height=126, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.view.ViewGroup$LayoutParams@7ae3216, tag=Father883, root-is-layout-requested=false, has-input-connection=false, x=21.0, y=21.0, child-count=1} ****MATCHES****
|
+------>ReactViewGroup{id=64, visibility=VISIBLE, width=84, height=84, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.view.ViewGroup$LayoutParams@b723797, tag=Son883, root-is-layout-requested=false, has-input-connection=false, x=21.0, y=21.0, child-count=1} ****MATCHES****
|
+------->ReactViewGroup{id=63, visibility=VISIBLE, width=42, height=42, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.view.ViewGroup$LayoutParams@25bdd84, tag=Grandson883, root-is-layout-requested=false, has-input-connection=false, x=21.0, y=21.0, child-count=0} ****MATCHES****
|
+---->ReactTextView{id=68, visibility=VISIBLE, width=129, height=51, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@6b3b96d, tag=ProductId000, root-is-layout-requested=false, has-input-connection=false, x=203.0, y=1003.0, text=Product, input-type=0, ime-target=false, has-links=false} ****MATCHES****
|
+---->ReactTextView{id=70, visibility=VISIBLE, width=129, height=51, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@45359a2, tag=ProductId001, root-is-layout-requested=false, has-input-connection=false, x=385.0, y=1003.0, text=Product, input-type=0, ime-target=false, has-links=false} ****MATCHES****
|
+---->ReactTextView{id=73, visibility=VISIBLE, width=129, height=51, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@218b633, tag=ProductId002, root-is-layout-requested=false, has-input-connection=false, x=566.0, y=1003.0, text=Product, input-type=0, ime-target=false, has-links=false} ****MATCHES****
|
+---->ReactTextView{id=75, visibility=VISIBLE, width=129, height=51, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@a7b4df0, tag=ProductId003, root-is-layout-requested=false, has-input-connection=false, x=748.0, y=1003.0, text=Product, input-type=0, ime-target=false, has-links=false} ****MATCHES****
|
+---->ReactViewGroup{id=79, visibility=VISIBLE, width=89, height=103, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=true, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@fdf7f69, tag=null, root-is-layout-requested=false, has-input-connection=false, x=496.0, y=1133.0, child-count=1} ****MATCHES****
|
+----->ReactTextView{id=78, visibility=VISIBLE, width=89, height=51, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.view.ViewGroup$LayoutParams@f316dee, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, text=Index, input-type=0, ime-target=false, has-links=false} ****MATCHES****
|
+---->ReactViewGroup{id=83, visibility=VISIBLE, width=89, height=104, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=true, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@b9e828f, tag=null, root-is-layout-requested=false, has-input-connection=false, x=496.0, y=1236.0, child-count=1} ****MATCHES****
|
+----->ReactTextView{id=82, visibility=VISIBLE, width=89, height=51, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.view.ViewGroup$LayoutParams@7df591c, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, text=Index, input-type=0, ime-target=false, has-links=false} ****MATCHES****
|
+---->ReactViewGroup{id=86, visibility=VISIBLE, width=89, height=103, has-focus=false, has-focusable=true, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=true, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@ee7a925, tag=null, root-is-layout-requested=false, has-input-connection=false, x=496.0, y=1340.0, child-count=1} ****MATCHES****
|
+----->ReactTextView{id=85, visibility=VISIBLE, width=89, height=51, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.view.ViewGroup$LayoutParams@5ca7afa, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0, text=Index, input-type=0, ime-target=false, has-links=false} ****MATCHES****
|
+->View{id=16908336, res-name=navigationBarBackground, visibility=VISIBLE, width=1080, height=126, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@c3be35b, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=1794.0} ****MATCHES****
|
+->View{id=16908335, res-name=statusBarBackground, visibility=VISIBLE, width=1080, height=63, has-focus=false, has-focusable=false, has-window-focus=true, is-clickable=false, is-enabled=true, is-focused=false, is-focusable=false, is-layout-requested=false, is-selected=false, layout-params=android.widget.FrameLayout$LayoutParams@827b7f8, tag=null, root-is-layout-requested=false, has-input-connection=false, x=0.0, y=0.0} ****MATCHES****
|
simonracz commented 6 years ago

To test android, please run mocha directly like this:

node_modules/.bin/mocha ./e2e --opts e2e/mocha.opts --configuration android.emu.debug --grep :ios: --invert

The reason is that some tests are ios specific, marked as :ios:. These should be excluded on android.

simonracz commented 6 years ago

As for iOS, I am not the best one to address it. But as far as I know, Earlgrey has trouble dealing with the soft keyboard sometimes.

rotemmiz commented 6 years ago

exactly what @simonracz said. make sure the mac keyboard layout is English. We still need to figure out how to run platform specific tests, the current way is to do what Simon says 😜

simonracz commented 6 years ago

I think this can be safely closed.