wix / Detox

Gray box end-to-end testing and automation framework for mobile apps
https://wix.github.io/Detox/
MIT License
11.2k stars 1.92k forks source link

Matching by.ID on conditionally rendered elements after a tap doesn't seem to work only on Text elements (iOs) #4051

Closed Omicron-Z closed 1 year ago

Omicron-Z commented 1 year ago

What happened?

Greetings everyone. I have been trying Detox on a clean react-native app on iOs, and I've run into an issue with the Matchers, both with detox 20.8 and with 20.9. Specifically, attempting to match by testID on a Text node which is rendered dynamically (the usual conditional rendering according to React status) after a tap doesn't work. However, the same match by testID does work on View nodes (in the same circumstances) or Text nodes statically rendered.

Put it in another way, trying to do:

    const helloButton = await element(by.id('hello_button'));
    await helloButton.tap();
    const helloText = await element(by.id('hello_text_view'));
    await expect(helloText).toExist();

Works. However, if I try to do the same on a

<Text testID="world_text">...World!!</Text>

I run into the following error:

    Test Failed: Failed expectation: TOEXIST WITH MATCHER(id == “world_text”)

    44 |     let worldText = await element(by.id('world_text'));
    > 45 |     await expect(worldText).toExist();

I also tried with

    await waitFor(element(by.id('world_text'))).toBeVisible().withTimeout(5000);

And in this case i get a timeout, which I haven't been able to solve. It seems excessive, a 5second out of sync error for a simple device tap. And why would it work with a View but not with a Text? You'll find the detailed code and logs below.

I might be missing something, but I've followed the tutorial, the documentation and the synch issue help page and couldn't get why it shouldn't work.

What was the expected behaviour?

I expect matchers to be able to select elements by testID regardless of them being Text or View elements, and to be able to do so without going in timeout after a single tap.

So, I'd expect the tests detailed above and below to be able to pass.

Was it tested on latest Detox?

Did your test throw out a timeout?

Help us reproduce this issue!

Example code of my React Native starting app for this test:

<Section testID="welcome" title="Detox Testing">
            <View testID="find_me">
              <Text testID="inner_text">Find Me Please!</Text>
            </View>
            {'\n'}
            <Button
              testID="hello_button"
              title="HELLO..."
              onPress={() => setHello(!hello)}
            />
            {'\n'}
            {hello && <View testID="hello_text_view"><Text>Hello...</Text></View>}
            {'\n'}
            <Button
              testID="world_button"
              title="WORLD!"
              onPress={() => setWorld(!world)}
            />
            {'\n'}
            {world && <Text testID="world_text">...World!!</Text>}
            {'\n'}
 </Section>

These three tests are passing correctly:

  it('should be able to find a Text element by id and check for their content', async () => {
    const innerFindMe = await element(by.id('inner_text'));
    await expect(innerFindMe).toExist();
    await expect(innerFindMe).toHaveText('Find Me Please!');
  });

  it('should not show the activatable texts', async () => {
    const helloText = await element(by.id('hello_text'));
    const worldText = await element(by.id('world_text'));
    await expect(helloText).not.toExist();
    await expect(worldText).not.toBeVisible();
  })

  it('should show hello screen after tap', async () => {
    const helloButton = await element(by.id('hello_button'));
    await helloButton.tap();
    const helloText = await element(by.id('hello_text_view'));
    await expect(helloText).toExist();
  });

However, this other never passes, in any way or form:

  it('should show world screen after tap', async () => {
    let worldButton = await element(by.id('world_button'));
    await worldButton.tap();
    // BUG?: Neither of these two solutions seem to be working. Why?
    // let worldText = await element(by.id('world_text'));
    // await expect(worldText).toExist();
    await waitFor(element(by.id('world_text'))).toBeVisible().withTimeout(5000);
    // NOTE:
    // Somehow it seems from the logger that the testID attribute as ax.id is missing from the worldText...
  });

In what environment did this happen?

Detox version: 20.9.0 React Native version: 0.70.7 Has Fabric (React Native's new rendering system) enabled: no Node version: 16.20.0 Device model: iPhone 14 iOS version: 16.2 macOS version: 13.3.1 Xcode version: 14.2 Test-runner (select one): jest

Detox logs

detox.log

Device logs

device.log

More data, please!

Test Start Screenshot:

Test Failure Screenshot:

d4vidi commented 1 year ago

@Omicron-Z if you wrap Text with View and set the test ID on the view - does it work?

<View testID="world_text"><Text>...World!!</Text></View>
d4vidi commented 1 year ago

Actually, before doing so, could you first run detox test and add the --capture-view-hierarchy=enabled argument? Please share with us the resulted files under .hierarchy/ under artifacts/ 🙏🏻

d4vidi commented 1 year ago

@asafkorem fyi

Omicron-Z commented 1 year ago

Hi d4vidi, first of all thanks a lot for your answers.

@Omicron-Z if you wrap Text with View and set the test ID on the view - does it work?

<View testID="world_text"><Text>...World!!</Text></View>

It does, actually I was already trying to do that test and that's also why I opened an Issue, because it seemed such an unintended behaviour and not a generic synchronization issue.

This section of the test app:

            {hello && <View testID="hello_text_view"><Text>Hello...</Text></View>}

Is covered successfully by this test:

  it('should show hello screen after tap', async () => {
    const helloButton = await element(by.id('hello_button'));
    await helloButton.tap();
    const helloText = await element(by.id('hello_text_view'));
    await expect(helloText).toExist();
  });

So I would have expected the same to work without the View wrapper.

Actually, before doing so, could you first run detox test and add the --capture-view-hierarchy=enabled argument? Please share with us the resulted files under .hierarchy/ under artifacts/ 🙏🏻

Done, here are the results (I had to zip them or Github won't allow the upload): ui.viewhierarchy.zip

d4vidi commented 1 year ago

I suspected this and it's likely right - Seems that React Native optimizes the view hierarchy and in fact concatenates nested texts (...World! and all the various \n's in our case). It so happens that the test ID is omitted, in the process. So as a workaround, I would stick with the wrapper View. It's unfortunate that this is the case but we can't control RN.

image
Omicron-Z commented 1 year ago

@d4vidi Thanks for the detailed follow-up, much appreciated.

I tried a couple of other approaches, just for completion. One was removing all the various {/n} and leave only the text, but that didn't change the results.

The other was wrapping the text with a view outside the conditionally rendered part. So, something like this:


            {'\n'}
            {/* See issue https://github.com/wix/Detox/issues/4051 for further details */}
            <View>{world && <Text testID="world_text">...World!!</Text>}</View>
            {'\n'}

That was enough to make the test pass.

So, to summarize, doing either of these workarounds

          <Button
            testID="hello_button"
            title="HELLO..."
            onPress={() => setHello(!hello)}
          />
          {'\n'}
          {hello && (
            <View testID="hello_text_view">
              <Text>Hello...</Text>
            </View>
          )}
          {'\n'}
          <Button
            testID="world_button"
            title="WORLD!"
            onPress={() => setWorld(!world)}
          />
          {'\n'}
          {/* See issue https://github.com/wix/Detox/issues/4051 for further details */}
          <View>{world && <Text testID="world_text">...World!!</Text>}</View>
          {'\n'}

Works and allows tests to behave as expected.

It might be worthy to add some kind of warning or disclaimer in the official docs.

Thanks a lot for clarification.

asafkorem commented 1 year ago

Interesting case @Omicron-Z, I'm glad it's resolved (@d4vidi You are the best! 👑)

@Omicron-Z We will consider your suggestion to add a disclaimer, though this bug / bug-by-design originates from react native. This optimization caused a disruption in their API so my suggestion is to open an issue on their repository, as they should include a disclaimer regarding this matter.