[Bug] Remote Config FetchAsync throws exception with TimeSpan

Nyankoo commented 2 years ago

[REQUIRED] Please fill in the following fields:

Unity editor version: 2020.3.18f1
Firebase Unity SDK version: 8.1.0
Source you installed the SDK: Unity Package Manager
Problematic Firebase Component: Remote Config
Other Firebase Components in use: Firestore, Crashlytics, Auth
Additional SDKs you are using:
Platform you are using the Unity editor on: Windows
Platform you are targeting: Android (Unity Editor is also affected)
Scripting Runtime: IL2CPP

[REQUIRED] Please describe the issue here:

When setting the cache expiration of FetchAsync() to TimeSpan.Zero (or any other TimeSpan), an exception is thrown with ErrorCode 1. The Message parameter is empty.

Steps to reproduce:

Relevant Code:

await FirebaseRemoteConfig.DefaultInstance.FetchAsync(TimeSpan.Zero); //This (or any other TimeSpan) will result in an exception

paulinon commented 2 years ago

Hi @Nyankoo,

Thanks for reporting this issue. Version 8.2.0 of the SDK has been released recently. Could you verify if the issue persists there as well? If it does, please provide a minimal, reproducible example of your project along with the complete steps to replicate this behavior.

Nyankoo commented 2 years ago

@paulinon Did something in Version 8.2.0 change related to Remote Config? The changelog doesn't mention anything.

paulinon commented 2 years ago

Hi @Nyankoo,

While the release notes don't mention anything related to Remote Config, this is to ensure that the issue persists even with the latest updates and fixes.

I look forward to your findings.

DanForstinger commented 2 years ago

I can confirm we are seeing this issue, on 8.2.0. It's causing our app to fully hang.

We are sending in a timespan of 1 minute. We started seeing this about 4 weeks ago. Can confirm removing the timespan argument stopped the issue from happening, but we can't wait for ~12 hours for remote config updates!

Should be simple enough to reproduce in an empty project on your end.

Nyankoo commented 2 years ago

@paulinon We didn't had a change to update, but Dan's report shows that this is also happening with 8.2.0.

DanForstinger commented 2 years ago

Also note @paulinon in our tests, this happened about 2/10 times-ish.

DanForstinger commented 2 years ago

If we remove the cache expiry, it works 100% of the time.

paulinon commented 2 years ago

Thanks for the input, @Nyankoo and @DanForstinger. So far, I have not encountered the exception when using the FetchAsync() function. A minimal, reproducible example would be helpful in identifying what's causing the exception.

Could you also confirm if the same exception persists using the quickstart?

DanForstinger commented 2 years ago

A few notes that may help you reproduce:

This was happening only on device, about ~10-20% of the time. Youll have to try a few times on device to reproduce.
It only happens if you call FetchAsync with a timespan, for example: FetchAsync(TimeSpan.FromMinutes(1)) is what our call was. FetchAsync() with no arguments works fine.

DanForstinger commented 2 years ago

For us, the app just hangs and never calls back from the async function, though we may not have got the errors because the initialization wasn't finishing on the main thread.

fonograph commented 2 years ago

To clarify, the situation Dan is describing is only happening after building on certain machines (CI vs local build), and only shows up at all for some % of users (some people don't see it at all.) It also started appearing randomly for an existing game we'd launched and made no changes to — which actually seems to suggest this is something happening on the firebase server side, rather than the unity side.

Nyankoo commented 2 years ago

Just posting to put the correct labels, as everything was already described by Dan and fonograph.

paulinon commented 2 years ago

Hi @Nyankoo, @DanForstinger, and @fonograph,

I have called the function on both an Android build and the Unity editor at least twenty times, but none of them resulted into an exception or issue.

There's a possibility that the issue is related to the device used. In order to verify if this is the case, could you provide the device models along with their respective Android versions?

If the issue doesn't persist when using the quickstart, it would be helpful if you provide a minimal, reproducible example of a project with this issue

Nyankoo commented 2 years ago

@paulinon Here are some more exception details (tested with 8.2.0 directly in the Unity editor):

FirebaseException
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () (at <695d1cc93cca45069c528c15c9fdd749>:0)
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) (at <695d1cc93cca45069c528c15c9fdd749>:0)
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) (at <695d1cc93cca45069c528c15c9fdd749>:0)
System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) (at <695d1cc93cca45069c528c15c9fdd749>:0)
System.Runtime.CompilerServices.TaskAwaiter.GetResult () (at <695d1cc93cca45069c528c15c9fdd749>:0)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1:Run() (at Library/PackageCache/com.cysharp.unitask@60bfbae787/Runtime/CompilerServices/StateMachineRunner.cs:189)
System.Threading.Tasks.TaskCompletionSource`1:SetException(Exception)
Firebase.<GetTask>c__AnonStorey0:<>m__0() (at Z:/tmp/tmp.Ti0ckjuoMn/firebase/app/client/unity/proxy/FutureVoid.cs:84)
Firebase.FutureVoid:SWIG_CompletionDispatcher(Int32) (at Z:/tmp/tmp.Ti0ckjuoMn/firebase/app/client/unity/proxy/FutureVoid.cs:167)
Firebase.AppUtilPINVOKE:PollCallbacks()
Firebase.AppUtil:PollCallbacks() (at Z:/tmp/tmp.Ti0ckjuoMn/firebase/app/client/unity/proxy/AppUtil.cs:32)
Firebase.Platform.FirebaseAppUtils:PollCallbacks() (at Z:/tmp/tmp.Ti0ckjuoMn/firebase/app/client/unity/proxy/FirebaseAppUtils.cs:33)
Firebase.Platform.FirebaseHandler:Update() (at Z:/tmp/tmp.xDmPRxPPw2/firebase/app/client/unity/src/Unity/FirebaseHandler.cs:208)
Firebase.Platform.FirebaseMonoBehaviour:Update() (at Z:/tmp/tmp.xDmPRxPPw2/firebase/app/client/unity/src/Unity/FirebaseMonoBehaviour.cs:45)

paulinon commented 2 years ago

Hi @Nyankoo,

Thanks for the additional information, but I am still unable to replicate the issue you're facing. At this point, we'll be needing a minimal, reproducible example of a project that exhibits this behavior. You may upload this in a GitHub repository and add me as a collaborator.

If the issue can be reproduced in a blank project, it would be helpful if you could provide a sample code along wih the complete steps needed to replicate the issue.

I look forward to hearing from you soon.

google-oss-bot commented 2 years ago

Hey @Nyankoo. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

Nyankoo commented 2 years ago

@paulinon We're not able to reproduce this in a separate project. Any ideas to get more details on this from our project?

paulinon commented 2 years ago

Hi @Nyankoo,

Is it possible to upload your project (without any PII) to a GitHub repository and add me as a collaborator?

Nyankoo commented 2 years ago

@paulinon That's sadly not possible as it is a company project.

DellaBitta commented 2 years ago

Hi @Nyankoo,

Since we haven't been able to reproduce this in our tests, and that you have a feeling that this might be server-side related since the code hasn't recently changed, I recommend opening up a Firebase Support Ticket. Firebase support can work to monitor the backend to see if any errors are occuring there, for instance.

Be sure to reference this ticket so that the support engineer has some background on what was tried, etc.

I hope this helps!

Nyankoo commented 2 years ago

@DellaBitta @paulinon We just got some iOS reports from Crashlytics in (from users, so no repo from us!), where the "Message" parameter of the exception is not null. This could be related and I'm not sure if this is caused by the SDK or the server:

Failed to get installations token. Error : Error Domain=com.firebase.installations Code=0 "Underlying error: The request timed out." UserInfo={NSLocalizedFailureReason=Underlying error: The request timed out., NSUnderlyingError=0x2819a2550 {Error Domain=NSURLErrorDomain Code=-1001 "The request timed out." UserInfo={_kCFStreamErrorCodeKey=-2102, NSUnderlyingError=0x281b0a2e0 {Error Domain=kCFErrorDomainCFNetwork Code=-1001 "(null)" UserInfo={_kCFStreamErrorCodeKey=-2102, _kCFStreamErrorDomainKey=4}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <DD48035E-BF2D-498A-8768-F529620D7412>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
"LocalDataTask <DD48035E-BF2D-498A-8768-F529620D7412>.<1>"
), NSLocalizedDescription=The request timed out., NSErrorFailingURLStringKey=https://firebaseinstallations.googleapis.com/v1/projects/my-project-id/installations/, NSErrorFailingURLKey=https://firebaseinstallations.googleapis.com/v1/projects/my-project-id/installations/, _kCFStreamErrorDomainKey=4}}}.

Failed to get installations token. Error : Error Domain=com.firebase.installations Code=0 "Underlying error: A data connection is not currently allowed." UserInfo={NSLocalizedFailureReason=Underlying error: A data connection is not currently allowed., NSUnderlyingError=0x281e784e0 {Error Domain=NSURLErrorDomain Code=-1020 "A data connection is not currently allowed." UserInfo={_kCFStreamErrorCodeKey=50, NSUnderlyingError=0x281e7b8a0 {Error Domain=kCFErrorDomainCFNetwork Code=-1020 "(null)" UserInfo={_kCFStreamErrorCodeKey=50, _kCFStreamErrorDomainKey=1}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <516FC515-5253-4C61-9EFE-D5143277783D>.<2>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
"LocalDataTask <516FC515-5253-4C61-9EFE-D5143277783D>.<2>"
), NSLocalizedDescription=A data connection is not currently allowed., NSErrorFailingURLStringKey=https://firebaseinstallations.googleapis.com/v1/projects/my-project-id/installations/, NSErrorFailingURLKey=https://firebaseinstallations.googleapis.com/v1/projects/my-project-id/installations/, _kCFStreamErrorDomainKey=1}}}.

Failed to get installations token. Error : Error Domain=com.firebase.installations Code=0 "Underlying error: The network connection was lost." UserInfo={NSLocalizedFailureReason=Underlying error: The network connection was lost., NSUnderlyingError=0x2814ea310 {Error Domain=NSURLErrorDomain Code=-1005 "The network connection was lost." UserInfo={NSUnderlyingError=0x2814ea670 {Error Domain=kCFErrorDomainCFNetwork Code=-1005 "(null)" UserInfo={_kCFStreamErrorCodeKey=54, _kCFStreamErrorDomainKey=1}}, NSErrorFailingURLStringKey=https://firebaseinstallations.googleapis.com/v1/projects/my-project-id/installations/, NSErrorFailingURLKey=https://firebaseinstallations.googleapis.com/v1/projects/my-project-id/installations/, _kCFStreamErrorDomainKey=1, _kCFStreamErrorCodeKey=54, NSLocalizedDescription=The network connection was lost.}}}.

Nyankoo commented 2 years ago

I can also confirm that this is reproduceable in our project only on a Windows machine (Android build target). On a Mac (iOS build target), it's working correctly.

Nyankoo commented 2 years ago

@DellaBitta @paulinon Another update: I noticed that once a day, the Fetch call works as expected, but when restarting the app directly after that, the error appears on every app launch until the next day. This could indicate that even when passing a TimeSpan to the Fetch call, the default 12 hour cache expiration is used and works, while all other times (using the passed TimeSpan) trigger the exception.

DellaBitta commented 2 years ago

Hi @Nyankoo,

I wanted to check in with you to see how things are going. I tried again to reproduce this locally by building on Windows and deploying on Android and unfortunately I had no luck in replicating the issue.

Have you opened a Firebase Support ticket to see if there's more logging on the backend?
The note about the Installations token is intriguing. Is this logging always present on RemoteConfig calls which never return? Additionally are you seeing similar logs on Android?

Thanks!

google-oss-bot commented 2 years ago

Hey @Nyankoo. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

google-oss-bot commented 2 years ago

Since there haven't been any recent updates here, I am going to close this issue.

@Nyankoo if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.

flogavel commented 2 years ago

Hi all, I have the same issue..

FirebaseRemoteConfig.FetchAsync(TimeSpan) sometimes (25 to 50% of the time) never returns a result on iOS devices (works well on android devices). Using FirebaseRemoteConfig.FetchAsync() solves the issue but we just can't afford to have a 12hours cache in our app.

Bug occurs on all our iOS devices.

Unity editor version: 2020.3.18f1 (also tried some 2019 versions) Firebase Unity SDK version: 7.1.0 Source you installed the SDK: Unity Package Manager Problematic Firebase Component: Remote Config Other Firebase Components in use: Firestore, Crashlytics, Auth Additional SDKs you are using: Platform you are using the Unity editor on: Mac Platform you are targeting: iOS Scripting Runtime: IL2CPP

Here's a code snippet if that can help :

// this method is called in the Start function of our main scene (but several scenes come before that one)
 void InitializeFirebaseRemoteConfig()
        {
            // These are the values that are used if we haven't fetched data from the
            // server
            // yet, or if we ask for values that the server doesn't have:

            _FirebaseRemoteConfig = FirebaseRemoteConfig.DefaultInstance;

            Dictionary<string, object> defaults = new Dictionary<string, object>
            {
                { "config_test_string", "default local string" },
                { "config_test_int", 1 },
                { "config_test_float", 1.0 },
                { "config_test_bool", false }
            };

            _FirebaseRemoteConfig.SetDefaultsAsync(defaults).ContinueWithOnMainThread(fetchTask1 =>
            {
                _FirebaseRemoteConfig.FetchAsync(new TimeSpan(0, 0, 10)).ContinueWithOnMainThread(fetchTask2 =>
                {
                    // doing our stuff here
                    // 25-50% of the time on iOS, we can't get here
                });
            });
        }

guerwan commented 2 years ago

@google-oss-bot we are still having this issue, detailed information provided in the comment above, could you please reopen this issue?

Nyankoo commented 2 years ago

@DellaBitta @paulinon Please re-open the issue, thanks!

DellaBitta commented 2 years ago

Hi @flogavel,

Thanks for the test case. I was able to reproduce the issue by replacing the quickstart code for InitializeFirebase with your implementation:

void InitializeFirebase()
{
      System.Collections.Generic.Dictionary<string, object> defaults = new System.Collections.Generic.Dictionary<string, object>
      {
          { "config_test_string", "default local string" },
          { "config_test_int", 1 },
          { "config_test_float", 1.0 },
          { "config_test_bool", false }
      };

Firebase.RemoteConfig.FirebaseRemoteConfig.DefaultInstance.SetDefaultsAsync(defaults).ContinueWithOnMainThread(fetchTask1 =>
      {
        DebugLog("RemoteConfig configured and ready!");
        isFirebaseInitialized = true;
        Firebase.RemoteConfig.FirebaseRemoteConfig.DefaultInstance.FetchAsync(new TimeSpan(0, 0, 10)).ContinueWithOnMainThread(fetchTask2 =>
          {
            DebugLog("Fetch Returned");
          });
      });
}

Tested this on iOS built with Unity 2020.3.20f1 LTS.

Marking this as a bug, thank you!

tinyrickguy commented 2 years ago

For those who are having this issue, we were able to resolve it by making sure that there are ZERO calls to the Firebase SDK before it has finished initializing. In our case, we were calling FirebaseRemoteConfig.DefaultInstance.Keys before CheckAndFixDependenciesAsync has finished calling its callback, which was the trigger to the broken behaviour described here. Not sure that it's the exact same issue others are having, but it did help us.

Nyankoo commented 2 years ago

We're awaiting CheckAndFixDependenciesAsync and initiate all other calls to the Firebase SDK after it returns with DependencyStatus.Available, so it seems this is unrelated.

jonsimantov commented 2 years ago

For those experiencing the issue where Fetch() hangs on iOS, please see https://github.com/firebase/quickstart-unity/issues/1198#issuecomment-1040690857 for more information. For Android, we are still looking into the issue.

a-maurice commented 2 years ago

Hey all,

Firebase SDK for Unity 8.9.0 is now live, which contains a fix for this issue, and you can download it directly here, or through the usual places like the package manager.

I'm going to close this issue for now, but please let us know if there are any outstanding problems.

Nyankoo commented 2 years ago

@a-maurice Just tested with 8.9.0, and the original issue is still happening when we call FirebaseRemoteConfig.DefaultInstance.FetchAsync(TimeSpan.Zero)

Nyankoo commented 2 years ago

@a-maurice @DellaBitta Please re-open this issue to continue to keep track of it.

a-maurice commented 2 years ago

@Nyankoo are you still seeing it in both Android and the editor?

Nyankoo commented 2 years ago

@a-maurice This currently only seems to happen in the Editor (tested on Android, not iOS), but I will update here when we released an update to our app and see if the issue still appears in Crashlytics.

Nyankoo commented 2 years ago

@a-maurice We're still seeing this in Crashlytics happening to our users, so it doesn't seem to be fixed on Android.

firebase / quickstart-unity