[FTL] `Firestore.permissionDenied` on attempt `setData` after success phone auth

dterekhov commented 1 year ago

Description

During UITests (XCTest), I constantly get FirestoreErrorCode.Code.permissionDenied on the attempt to docRef.setData(entityDict) - it's creating a new user in the root collection. It happens after successful auth in Firebase. Authentication lib. My Firestore's security rules are totally disabled for the sake of debugging:

rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /{document=**} {
      allow read, write: if true;
    }
  }
}

Interestingly enough, I always successfully run the same UITest locally in Xcode (and in Terminal too). In the Test lab, I run a test on iPhone 11 Pro, iOS 16.3. We also use AppCheck. But it perfectly works in TestFlight, AppStore, and locally in Xcode for a long time. But not in the Firebase Test lab that I trying to use.

It's a video recorded by FTL after failed UI test, to give a better sense of what's happening. After the failure, I printed on our Start screen all steps that were done. Please help! I'd very much like to use the Firebase Test lab.

Reproducing the issue

No response

Firebase SDK Version

10.9

Xcode Version

14.3

Installation Method

Swift Package Manager

Firebase Product(s)

Analytics, App Check, Authentication, Crashlytics, DynamicLinks, Firestore, Messaging, Remote Config, Storage

Targeted Platforms

iOS

Relevant Log Output

No response

If using Swift Package Manager, the project's Package.resolved

Expand Package.resolved snippet

```json Replace this line with the contents of your Package.resolved. ```

If using CocoaPods, the project's Podfile.lock

Expand Podfile.lock snippet

```yml Replace this line with the contents of your Podfile.lock! ```

milaGGL commented 1 year ago

Hi @dterekhov, thank you for reporting this issue. What I understand is you are able to create a new user locally with Xcode, but the same code is failing with permissionDenied error code on Test lab, correct?

Is it only the mutations like setData is failing, or all interactions with Firestore are denied? Can you read data from Firestore?

Does the firebaseUser_did id represents the new user's ID you just created? If yes, then the permissionDenied error is actually received after a user is created, right? Can we check which Firestore request was rejected?

Would it be possible to provide a reproduction code for us to reproduce the bug?

dterekhov commented 1 year ago

you are able to create a new user locally with Xcode, but the same code is failing with permissionDenied error code on Test lab, correct

Correct.

Is it only the mutations like setData is failing, or all interactions with Firestore are denied? Can you read data from Firestore?

Only setData is failing.

the permissionDenied error is actually received after a user is created, right?

Correct, permissionDenied is throw after FirebaseAuth user creation. Specificially, on setData step.

Would it be possible to provide a reproduction code for us to reproduce the bug?

Let me provide more info that could be helpful.

Confirmation if you're also encountering the issue using other devices aside from iPhone 11 Pro, iOS 16.3

Yes, it fails on other iPhones too that are available in the Test lab. I checked it.

Sample XCTest package (.zip) you’ve uploaded that reproduces the issue

It's here in the archive.

Matrix ID from the Firebase Test Lab console when you ran the test which shows the issue (the matrix ID should start with "matrix-“)

Test Lab > Delight > matrix-3qxgwwjknzqqj

Detailed or step-by-step instructions to reproduce the issue:

XCUITest robot presses “Continue with Phone Number”
Inputs test phone number +19491112222, presses “Next”.
Fills verification text field with test code 123456, and presses “Done”. In this last step FirebaseAuth user is created, and when in code isCurrentFIRUserAuthenticated global variable flag becomes true in the snippet: Auth.auth().addStateDidChangeListener { (_, user) in self.isCurrentFIRUserAuthenticated = user != nil } I create a userProfile document in our root Firestore collection with the method:

do {
   try await Firestore.firestore().collection(“userProfiles").setData(entityDict)
} catch {
   /* Get `FirestoreErrorCode.Code.permissionDenied` every time */
}

On video on throwing permissionDenied, the app goes back to the initial Welcome screen (with “Continue with Phone Number” button). On on this fail, I print out some step logs right over the Welcome screen. On the "run locally” video you can see it successfully goes next and continue to fill in the rest information. One more time: it runs successfully everywhere - locally in Xcode, in Terminal (XCUITest), TestFlight, and AppStore. Everywhere except the Firebase Test lab.

Video result showing the expected or working result that you've run locally on your machine

Run locally on my machine
Run remotely, video auto-generated by Test lab.

cc @milaGGL

Please lmk if I can help more to resolve this issue.

milaGGL commented 1 year ago

Hi @dterekhov, since the test case is passing locally, but not in test lab, my suspicion is that this discrepancy may be due to the config set ups in the test lab. Could you please help verify this by running the test on an actual iOS device, preferably the same one you have used in test lab.

dterekhov commented 1 year ago

Could you please help verify this by running the test on an actual iOS device

@milaGGL yes, it successfully runs and finishes on my real device. Please take a look at the attached video in my post above. It's where I say "Run locally on my machine" 👆

Unfortunately, I have no the same devices that available in the Test lab. But I tried - it fails on all provided iPhones in the Test lab.

milaGGL commented 1 year ago

Sorry for the confusion, I meant connecting an iPhone to your computer, and run the tests on the phone, instead of an iOS simulator.

dterekhov commented 1 year ago

Sorry for the confusion, I meant connecting an iPhone to your computer, and run the tests on the phone, instead of an iOS simulator.

Already did it. On real physical iOS device. It runs with the success on my Mac ✅ I even recorded a video how it goes above.

milaGGL commented 1 year ago

@dterekhov, since the local tests are passing, I am leaning toward it is something related to FTL. You have mentioned App Check in products list, it might be the reason. Would it be possible to isolate out App Check from your app and run the test again on FTL?

dterekhov commented 1 year ago

@milaGGL For the Simulator and UI tests I used AppCheck.setAppCheckProviderFactory(AppCheckDebugProviderFactory()) so it shouldn't be a problem there. Btw earlier I tried both prod and dev credentials for AppCheck + totally disabled it. Didn't help to run in the Test lab.

dterekhov commented 1 year ago

Hi @milaGGL, is there any update? It's 2 weeks silence on this issue. Just would like to know is this task in progress or not in work at all?

milaGGL commented 1 year ago

Hi @dterekhov, sorry for the silence, I should have updated the assignee.

Since the tests are successful on simulators and real devices but fails specifically on TestLab, I have reached out to a colleague for assistance. They will address the problem according to its priority.

Meanwhile, it would be greatly appreciated if you could help provide a minimal repro code to reproduce and debug the issue on our end.

adamvduke commented 1 year ago

Hi @dterekhov, I'm looking into this. Can you provide a more recent matrix id that demonstrates the same issue, or alternatively grant permission to re-run matrix-3qxgwwjknzqqj?

dterekhov commented 1 year ago

Hi @adamvduke. Yes, I can do it tomorrow.

jakeouellette commented 1 year ago

@dterekhov One additional question we have on your side if you have any details on the signing cert that might help us -- e.g., could you try signing with a different cert locally and run the same test and see if it fails?

(Apologies for this, because I'm /really/ unfamiliar with Firestore on iOS) but is there any additional debug logging you'd be able to provide us for each case? e.g., via something like Firestore.enableLogging(true), to make sure the authentication isn't happening in different orders locally / remotely / testflight / etc?

dterekhov commented 1 year ago

Hi @dterekhov, I'm looking into this. Can you provide a more recent matrix id that demonstrates the same issue, or alternatively grant permission to re-run matrix-3qxgwwjknzqqj?

@adamvduke, The new matrix is matrix-3l1jqmhq7c8t7. Just in case, a failed video generated by FTL.

dterekhov commented 1 year ago

@jakeouellette, another local cert works fine. Enabling Firestore.enableLogging(true) locally doesn't show potential issues as it runs smoothly locally before. But having this extra logs line enabled remotely (run with FTL), unfortunately I don't know the way how to see these extended logs or any logs (as it runs on a remote machine's Xcode). If I do then I don't bother you guys :)

One trick that I do, I print all occurred fails and errors right over the main app screen and check the video that generated automatically by FTL.

dterekhov commented 1 year ago

Any update guys?

brown-qs commented 1 year ago

This issue happens for years but it's still bothering developers.

[Error: [firestore/permission-denied] The caller does not have permission to execute the specified operation.]

My firestore rule:

rules_version = '2'; service cloud.firestore { match /databases/{database}/documents { match /{document=**} { allow read, write: if true; } } } Firebase SigninWithEmailAndPassword succeeds, but when firestore.collection('users').doc(uid).get() it displays this error.

"@react-native-firebase/app": "^18.5.0", "@react-native-firebase/auth": "^18.5.0", "@react-native-firebase/crashlytics": "^18.5.0", "@react-native-firebase/dynamic-links": "^18.5.0", "@react-native-firebase/firestore": "^18.5.0", note: it happens both on iOS and Android

Looking forward to answer soon!

firebase / firebase-ios-sdk