wix / Detox

Gray box end-to-end testing and automation framework for mobile apps
https://wix.github.io/Detox/
MIT License
11.25k stars 1.92k forks source link

Methods provided for testing Biometrics don't work #2981

Closed arish-panjwani closed 2 years ago

arish-panjwani commented 3 years ago

Description

I have implemented the Biometric Check Functionality(Face ID and Touch ID) in my App and I need to test it using Detox.

I have tried the methods provided in detox docs (https://github.com/wix/Detox/blob/7bed6ea8a2bef11576ca0014a4596776d9320cfd/docs/APIRef.DeviceObjectAPI.md#devicesetbiometricenrollmentbool-ios-only) but they don't seem to work.

I have used the following code in my test where the biometric is asked by the app.

it("matchFinger Test", async () => { await device.setBiometricEnrollment(true); await device.matchFinger(); }

and

it("matchFace Test", async () => { await device.setBiometricEnrollment(true); await device.matchFace(); }

Expected behavior

It should match the face ID/touch ID and login successfully.

Actual Output

The test gets stuck and the following output is given continuously:

detox[8939] INFO: [APP_STATUS] The system is busy with the following tasks:

Dispatch Queue ⏱ Queue: “Main Queue ()” with 1 work item

Run Loop ⏱ “Main Run Loop” detox[8939] INFO: [APP_STATUS] The system is busy with the following tasks:

Dispatch Queue ⏱ Queue: “Main Queue ()” with 1 work item

Run Loop ⏱ “Main Run Loop”

Screenshots

image

Environment:

tradebulls commented 3 years ago

We are facing the same issue. Any solution?

npjoge commented 3 years ago

I am also facing same kind of issues. Any solution?

arish-panjwani commented 3 years ago

@noomorph @d4vidi @LeoNatan Any Solutions for this?

noomorph commented 3 years ago

@alon-ha, could you find some time to look this week at this?

d4vidi commented 3 years ago

@alon-ha could you spare 1-2hr for this?

alon-ha commented 3 years ago

Will take a look this week

arish-panjwani commented 3 years ago

@alon-ha Any update on the issue?

alon-ha commented 3 years ago

Not yet, I'm in the effort of introducing XCUITest into the project. Will try to squeeze it in.

arish-panjwani commented 3 years ago

That's great @alon-ha, so can you please provide a timeline on when can we expect the update?

d4vidi commented 3 years ago

@arish-panjwani we have plans for a bugs burn-down week in about a month. I'll try to push that in beforehand, but that ought to be the latest we get to this.

arish-panjwani commented 2 years ago

@d4vidi @alon-ha Any update on this?

noomorph commented 2 years ago

@d4vidi @alon-ha Any update on this?

@alon-ha is no longer working with us on Detox.

@asafkorem howdy! Can you take a look at this issue? 🙏

asafkorem commented 2 years ago

Thanks @arish-panjwani for reporting the issue! I will investigate this issue very soon.

In the meanwhile, I'd appreciate if you could answer me a few questions:

  1. Did you called setBiometricEnrollment(true) before the authentication request was started?
  2. Did you called matchFinger() / matchFace() after the authentication request was started?
  3. In what point the test gets stuck?

Also, it would be very useful if you can reproduce this issue in a real app & test that you can share with me, you can use https://github.com/wix-incubator/DetoxTemplate if it helps. Alternatively please provide here the code that you are testing with the problematic tests (unless the tests code described above is enough for reproducing the issue).

arish-panjwani commented 2 years ago

Thanks for the reply @asafkorem, I'll check and reply in a couple days.

arish-panjwani commented 2 years ago

Thanks @arish-panjwani for reporting the issue! I will investigate this issue very soon.

In the meanwhile, I'd appreciate if you could answer me a few questions:

  1. Did you called setBiometricEnrollment(true) before the authentication request was started?
  2. Did you called matchFinger() / matchFace() after the authentication request was started?
  3. In what point the test gets stuck?
  • If the answer for 1 /2 is no, please try to change your code so it will fit the following flow: enable biomentric enrollment > invoke authentication request -> call the matching method.

Also, it would be very useful if you can reproduce this issue in a real app & test that you can share with me, you can use https://github.com/wix-incubator/DetoxTemplate if it helps. Alternatively please provide here the code that you are testing with the problematic tests (unless the tests code described above is enough for reproducing the issue).

@asafkorem I did call setBiometricEnrollment(true) before the authentication request was started and matchFinger() / matchFace() after the authentication request was started.

Firstly, setBiometricEnrollment(true) does not set Biometric Enrollment to true and I have to do it manually from Features -> Face ID -> Enrolled.

Secondly, even after doing that, the test gets stuck at matchFace() with the message "The system is busy"(please refer to the screenshot in the original issue statement) and I have to manually match the Face on Simulator by going to Features -> Face ID -> Matching Face.

Thus technically neither of the commands actually work.

Please look at the following code implementation:

import { logout, userLogin } from "@e2e/CommonTestCases";

describe("Biometrics Test", () => { beforeAll(async () => { await device.launchApp(); });

//Logs out the user if already logged in logout();

//Logs in the user and enables biometrics from inside the app userLogin(true);

//Supposed to set Biometric Enrollment to true it("Set Biometric Enrollment Test", async () => { await device.setBiometricEnrollment(true); });

//Kills the app and reopens it (Note: This time the login Id and password are not required but Pin or Biometrics is required as 2FA) it("Kill and reopen App", async () => { await device.terminateApp(); await device.launchApp(); });

// Supposed to Match Face it("matchFace Test", async () => { await device.matchFace(); }); });

arish-panjwani commented 2 years ago

@asafkorem any update on this?

asafkorem commented 2 years ago

Hi @arish-panjwani, thanks for the information :) First - what are logout() and userLogin()? why they are not in the beforeAll()?

Please help me investigate and reproduce this better, I suggest to write a small isolated test that does the minimal actions.

If I understand your flow correctly, it should be something like:

it("should match face on login, async () => {
  await device.launchApp();

  logout();

  await device.setBiometricEnrollment(true);
  login(); 

  await device.matchFace();
});

Can you please test if this works? please attach a recording of this test if not, you can add --artifacts-location <folder name> --record-videos failing in your test command (see here).

asafkorem commented 2 years ago

Also, if it still fails, and you want to expedite the investigation, please reproduce this issue on our template app, it will be very helpful.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this reporsitory, read this discussion.

stale[bot] commented 2 years ago

The issue has been closed for inactivity.

MuckT commented 2 years ago

Was able to get this working with react-native-touch-id here: DetoxTemplate-biometrics; however, I did run into some issues with the device.unmatch() timing out as I couldn't interact with the system dialog.

In attempting to implement biometrics I also found that the library react-native-keychain might not work on iOS simulators, so that might be part of the issue others are experiencing. https://github.com/oblador/react-native-keychain/issues/340#issuecomment-631681484

rikur commented 2 years ago

Same issue, running on iOS 16 simulator and latest detox. Subscribing to the GH issue.