Closed nitzanwilnai closed 1 year ago
The code to send data to Firestore is very simple:
public void SendUserDataToFireStore(UserData userData)
{
FirebaseTimeout = Constants.FIREBASE_TIMEOUT;
FirebaseFirestore db = FirebaseFirestore.DefaultInstance;
if (PlayerPrefs.GetInt(FirebaseUser.UserId) == 0)
PlayerPrefs.SetInt(FirebaseUser.UserId, 1);
DocumentReference docRef = db.Collection(Constants.UserDataCollection).Document(FirebaseUser.UserId);
docRef.SetAsync(userData).ContinueWith(task =>
{
if (task.IsCompleted)
{
Debug.Log("UserData DocRef ID " + docRef.Id);
}
else
{
Debug.Log("SendUserDataToFireStore ERROR ADDING DATA TO FIRESTORE!");
}
});
}
One more thing I'll add, we have two sets of data we are storing, the UserData and the LevelData, and I am calling them both at the same time. I am not waiting for the UserData to finish before calling SetAsync on the LevelData. Could this be the issue?
Hi,
Thank you for filling up the ticket! The team is not available for support during the holiday break and we will get back to you in the new year. Happy holiday!
Some additional information: Unity 2020.3.41f1 Firebase Unity SDK 10.1.1
Crashlytics reports -
Hi @nitzanwilnai ,
Could you provide the crash report in detail? For example what is the error message when crashes?
I am not sure what you are asking. I included all the information above. The messages in Google Play and Firebase and the stack traces. There is no error message, only the crash report. We know that on the user device in production the game simply exits.
Note that every single user successfully saved their data to Firestore multiple times. Most had 20+ minute sessions where they saved the data dozens of times without issue before the crash happened.
If it makes it easier I can combine all the info here:
Firebase:
Google Play:
pid: 0, tid: 17430 >>> com.fourxp.mergeup <<<
backtrace:
pid: 0, tid: 16730 >>> com.fourxp.mergeup <<<
backtrace:
pid: 0, tid: 11499 >>> com.fourxp.mergeup <<<
backtrace:
pid: 0, tid: 10167 >>> com.fourxp.mergeup <<<
backtrace:
pid: 0, tid: 14281 >>> com.fourxp.mergeup <<<
backtrace:
pid: 0, tid: 3325 >>> com.fourxp.mergeup <<<
backtrace:
We also believe that the non-firebase issues on Google Play are also related:
For example the various [libc.so]
abort and [libc.so] prop_area::map_prop_area_rw(char const*, char const*, bool*)
errors.
But none of them have any stack trace.
Note also that we have not been able to reproduce any of these issues locally, they only happen in production.
Note that updating to Firebase 10.3.0 did not fix the issue.
This is likely a duplicate of https://github.com/firebase/firebase-unity-sdk/issues/569 (the android global refs exhaustion issue). If you have a lot of Firestore documents in memory with a lot of key/value pairs and/or large arrays in them then this can exhaust the pool of 51200 global references, resulting in a crash. We are actively working on a fix for this (https://github.com/firebase/firebase-cpp-sdk/pull/1176) and hope to have it released in the next Unity SDK, planned for late January or early February 2023.
So our documents are fairly optimized (bytes instead of ints, single letter variable names, etc...) We are sending the same document fairly often. (the game data) We are also sending one way - we receive the game data at loading, afterwards we only send the new data. Is there a way to manually clear the pool?
I've looked at your crash reports more closely and can confirm 100% that you are experiencing the Android "global refs exhaustion issue" (https://github.com/firebase/firebase-unity-sdk/issues/569). The fact that the crashes are occurring in art::JNI::NewGlobalRef(...)
confirms this.
Now, it's not the size of the documents, in bytes, that matters, but rather the number of key value pairs and the number of elements in array and map values. Each value and each array/map element consumes a JNI global reference, along with each Firestore object (e.g. DocumentSnapshot, Transaction). Unfortunately, there really is no workaround other than reducing the number of Firestore objects in memory at a given time. If you have documents with large array or map values, then avoid keeping those documents in memory.
The upcoming fix will likely address this crash though. It will, unfortunately, be a month until the release of the fix though.
I'm going to close this issue since it is a duplicate of https://github.com/firebase/firebase-unity-sdk/issues/569. Please follow that issue for updates. We can still discuss here in this issue if you have follow-up quetsions.
Can you explain to me a bit more what you mean by keeping these documents in memory? Are you talking about the memory during the SetAsync call? When is this memory freed?
Or are you talking about the copy of my [FiretoreData] class?
From our logs, it seems like there is a memory leak somewhere in Firebase every time we send our FirestoreData class to firebase, and that after X amount of times we exceed the table you are talking about.
Thanks!
Oh, all I mean is holding on to DocumentSnapshot objects. Make sure you null out references to Firestore objects once you don't need them anymore (or let them go out of scope so they will be garbage collected). If your app is well-written, there may not be any opportunity here to do that though.
So in my SendUserDataToFireStore code here:
`
public void SendUserDataToFireStore(UserData userData)
{
FirebaseTimeout = Constants.FIREBASE_TIMEOUT;
FirebaseFirestore db = FirebaseFirestore.DefaultInstance;
if (PlayerPrefs.GetInt(FirebaseUser.UserId) == 0)
PlayerPrefs.SetInt(FirebaseUser.UserId, 1);
DocumentReference docRef = db.Collection(Constants.UserDataCollection).Document(FirebaseUser.UserId);
docRef.SetAsync(userData).ContinueWith(task =>
{
if (task.IsCompleted)
{
Debug.Log("UserData DocRef ID " + docRef.Id);
}
else
{
Debug.Log("SendUserDataToFireStore ERROR ADDING DATA TO FIRESTORE!");
}
});
}
`
Do I need to save a reference to the docRef, then set it to null once the ContinueWith returns?
The DocumentReference only consumes 1 global reference. So unless you have a LOT of DocumentReference objects alive at once then it doesn't matter. But to answer your question, you should set docRef to null ASAP, unless it just goes out of scope, which has the same effect as setting it to null. So probably the cruft of setting it to null is not worth it.
The more interesting object is the userData
object. How many key/value pairs does it contain? Does it contain any array or map values?
So yes, we have two big objects, userData and businessData, and they both have a few arrays, some of them quite large. In one we had an array with 1024 integers.
We also have nested arrays of other FirestoreData classes. For example we have our BusinessData class, which references BoardData, which has arrays of ProducerData, AutoMorphData, BlenderData, etc...
We have tried to reduce everything to as small as possible and so far are seeing fewer crashes.
That said, I still don't understand why the crashes happen after 20+ minutes of gameplay and dozens of SetAsync calls if the memory gets freed once the SetAsync call is done.
Here is our userData class: ` using System; using System.Collections; using System.Collections.Generic; using System.Runtime.InteropServices; using Firebase.Firestore; using UnityEngine;
namespace MergeGame { [FirestoreData] public class UserData { [FirestoreProperty] public int V { get; set; } // version [FirestoreProperty] public string AB { get; set; } // AB Test
[FirestoreProperty] public DateTime T1 { get; set; } // DateTime of First login event
[FirestoreProperty] public DateTime TC { get; set; } // DateTime of Last login event sent to analytics
[FirestoreProperty] public int G { get; set; } // Group, from 1-12. Will be used for A/B testing in the future
[FirestoreProperty] public string N { get; set; } // username
[FirestoreProperty] public bool A { get; set; } // admin
[FirestoreProperty] public int Gems { get; set; }
[FirestoreProperty] public short Energy { get; set; }
[FirestoreProperty] public long EnergyTime { get; set; }
[FirestoreProperty] public int[] DiscoveredFullIDs { get; set; }
[FirestoreProperty] public short DiscoveredCount { get; set; }
[FirestoreProperty] public short[] MiniTutorialChain { get; set; }
[FirestoreProperty] public short MiniTutorialChainCount { get; set; }
[FirestoreProperty] public int[] MiniTutorialItem { get; set; }
[FirestoreProperty] public short MiniTutorialItemCount { get; set; }
[FirestoreProperty] public bool MiniTutorialProducerRefill { get; set; }
[FirestoreProperty] public bool MiniTutorialEnergyRefill { get; set; }
[FirestoreProperty] public bool MTB { get; set; } // Mini tutorial bubble
[FirestoreProperty] public int DebugBundleCounter { get; set; }
[FirestoreProperty] public bool MuteSFX { get; set; }
[FirestoreProperty] public bool MuteMusic { get; set; }
[FirestoreProperty] public bool DisableLocalNotifications { get; set; }
[FirestoreProperty] public bool UserRatedApp { get; set; }
[FirestoreProperty] public int RateUsPopupLastOpenedLevel { get; set; }
}
} `
You can tell that at some point we actually read the Firestore doc and started changing our names to the shortest possible. ;) I assume name length does not affect this issue, only the size of the data stored?
Our BusinessData:
` using System; using System.Collections; using System.Collections.Generic; using System.Runtime.InteropServices; using Firebase.Firestore;
namespace MergeGame { [FirestoreData] public class ProducerData { [FirestoreProperty] public int ItemIdx { get; set; } [FirestoreProperty] public int ItemCounter { get; set; } [FirestoreProperty] public long CooldownStart { get; set; }
public ProducerData() { }
public ProducerData(int itemIdx, int itemCounter, long cooldownStart)
{
ItemIdx = itemIdx;
ItemCounter = itemCounter;
CooldownStart = cooldownStart;
}
}
[FirestoreData]
public class AutoMorphData
{
[FirestoreProperty] public int ItemIdx { get; set; }
[FirestoreProperty] public long AutoMorphStart { get; set; }
public AutoMorphData() { }
public AutoMorphData(int itemIdx, long autoMorphStart)
{
ItemIdx = itemIdx;
AutoMorphStart = autoMorphStart;
}
}
[FirestoreData]
public class ConverterData
{
[FirestoreProperty] public int ItemIdx { get; set; }
[FirestoreProperty] public int ConversionItemID { get; set; }
[FirestoreProperty] public int ConversionTime { get; set; }
[FirestoreProperty] public long ConversionStart { get; set; }
public ConverterData() { }
public ConverterData(int itemIdx, int conversionItemID, int conversionTime, long conversionStart)
{
ItemIdx = itemIdx;
ConversionItemID = conversionItemID;
ConversionTime = conversionTime;
ConversionStart = conversionStart;
}
}
[FirestoreData]
public class BlenderData
{
[FirestoreProperty] public int ItemIdx { get; set; }
[FirestoreProperty] public int FirstAddedItemID { get; set; }
[FirestoreProperty] public int ConversionItemID { get; set; }
[FirestoreProperty] public int ConversionTime { get; set; }
[FirestoreProperty] public long ConversionStart { get; set; }
public BlenderData() { }
public BlenderData(int itemIdx, int firstAddedItemID, int conversionItemID, int conversionTime, long conversionStart)
{
ItemIdx = itemIdx;
FirstAddedItemID = firstAddedItemID;
ConversionItemID = conversionItemID;
ConversionTime = conversionTime;
ConversionStart = conversionStart;
}
}
[FirestoreData]
public class BoardData
{
[FirestoreProperty] public short[] Indices { get; set; }
[FirestoreProperty] public short[] Items { get; set; }
[FirestoreProperty] public ProducerData[] Producers { get; set; }
[FirestoreProperty] public int ProducersCount { get; set; }
[FirestoreProperty] public int[] OrderProducers { get; set; }
[FirestoreProperty] public int OrderProducersCount { get; set; }
[FirestoreProperty] public AutoMorphData[] AutoMorphItems { get; set; }
[FirestoreProperty] public int AutoMorphItemsCount { get; set; }
[FirestoreProperty] public ConverterData[] ConverterItems { get; set; }
[FirestoreProperty] public int ConverterItemsCount { get; set; }
[FirestoreProperty] public BlenderData[] BlenderItems { get; set; }
[FirestoreProperty] public int BlenderItemsCount { get; set; }
[FirestoreProperty] public long FogOfWar { get; set; }
}
[FirestoreData]
public class BOrderData
{
[FirestoreProperty] public int Idx { get; set; } //index of current order in burst
[FirestoreProperty] public int ID { get; set; } //id of current order // NOT USED
[FirestoreProperty] public int ICnt { get; set; } //items count in current order
[FirestoreProperty] public short[] IIDs { get; set; } //item ids in current order
[FirestoreProperty] public long T { get; set; } //countdown time between bursts
[FirestoreProperty] public byte C { get; set; } // character
}
[FirestoreData]
public class OrderData
{
[FirestoreProperty] public int OrderID { get; set; }
[FirestoreProperty] public int NumItems { get; set; }
[FirestoreProperty] public short[] ItemIDs { get; set; }
[FirestoreProperty] public byte C { get; set; } // character
}
[FirestoreData]
public class BusinessData
{
[FirestoreProperty] public int V { get; set; } // version
[FirestoreProperty] public byte GameState { get; set; }
[FirestoreProperty] public byte PrevGameState { get; set; }
[FirestoreProperty] public bool VIP { get; set; }
[FirestoreProperty] public int XP { get; set; }
[FirestoreProperty] public int RoadmapLevel { get; set; }
[FirestoreProperty] public int RoadmapPrestige { get; set; }
[FirestoreProperty] public long[] FreeRewardsCollected { get; set; } // used as an array of 64 bit flags
[FirestoreProperty] public long[] VIPRewardsCollected { get; set; } // used as an array of 64 bit flags
[FirestoreProperty] public BoardData BD { get; set; }
[FirestoreProperty] public short[] UncollectedRewards { get; set; }
[FirestoreProperty] public short UncollectedRewardsCount { get; set; }
[FirestoreProperty] public OrderData[] OD { get; set; }
[FirestoreProperty] public int OrderCount { get; set; }
[FirestoreProperty] public int OrderUniqueID { get; set; }
[FirestoreProperty] public int CompletedOrderCycle { get; set; }
[FirestoreProperty] public int OrderSizeCycle { get; set; }
[FirestoreProperty] public sbyte PMOrdrIdx { get; set; }
[FirestoreProperty] public sbyte MetaOrderIdx { get; set; }
[FirestoreProperty] public sbyte MetaOrderCompleteIdx { get; set; }
[FirestoreProperty] public BOrderData BOrdr { get; set; }
[FirestoreProperty] public int[] FullItemIDs { get; set; }
[FirestoreProperty] public int HardTutorialStep { get; set; }
[FirestoreProperty] public int SoftTutorialStep { get; set; }
[FirestoreProperty] public int SoftTutorialAction { get; set; }
[FirestoreProperty] public bool RoadmapVisited { get; set; } // for RoadMap Auto Scroll
[FirestoreProperty] public short[] MetaPendingIDs { get; set; }
[FirestoreProperty] public byte[] MetaPendingCounters { get; set; }
[FirestoreProperty] public short MetaPendingCount { get; set; }
[FirestoreProperty] public short[] MetaDoneIDs { get; set; }
[FirestoreProperty] public short MetaDoneCount { get; set; }
[FirestoreProperty] public sbyte[] InitialOrderIdx { get; set; }
[FirestoreProperty] public short LastItemRemovedID { get; set; }
[FirestoreProperty] public byte LastItemRemovedBoardIdx { get; set; }
}
} `
Also here is a log from a user that crashed. You can see they sent the userData and businessData many times successfully before finally crashing on the last SendBusinessDataToFireStore call.
We only send the data once the user is inactive for 2.5sec. So as long as the user is interacting with the application we are not sending any data.
1 06:53:22.216 PM SendBusinessDataToFireStore - BOARD 2 06:53:22.018 PM SendUserDataToFireStore - B 3 06:52:36.397 PM SendBusinessDataToFireStore - BOARD 4 06:52:36.215 PM SendUserDataToFireStore - B 5 06:52:16.663 PM SendBusinessDataToFireStore - MINI_TUTORIAL_BUBBLE 6 06:52:16.481 PM SendUserDataToFireStore - B 7 06:52:13.966 PM ShowBubbleMiniTutoial boardIdx 38 8 06:51:52.514 PM SendBusinessDataToFireStore - BOARD 9 06:51:52.316 PM SendUserDataToFireStore - B 10 06:51:35.532 PM SendBusinessDataToFireStore - BOARD 11 06:51:35.301 PM SendUserDataToFireStore - B 12 06:51:09.872 PM SendBusinessDataToFireStore - BOARD 13 06:51:09.644 PM SendUserDataToFireStore - B 14 06:51:09.642 PM OnApplicationFocus(False) - B 15 06:51:09.611 PM SendBusinessDataToFireStore - BOARD 16 06:51:09.446 PM SendUserDataToFireStore - B 17 06:50:50.579 PM SendBusinessDataToFireStore - BOARD 18 06:50:50.381 PM SendUserDataToFireStore - B 19 06:50:35.175 PM SendBusinessDataToFireStore - BOARD 20 06:50:34.928 PM SendUserDataToFireStore - B 21 06:50:16.610 PM SendBusinessDataToFireStore - BOARD 22 06:50:11.600 PM SendUserDataToFireStore - B 23 06:50:11.600 PM OnApplicationFocus(False) - B 24 06:50:11.548 PM OnApplicationPause(True) - B 25 06:50:10.145 PM SendBusinessDataToFireStore - BOARD 26 06:50:09.824 PM SendUserDataToFireStore - B 27 06:50:09.824 PM OnApplicationFocus(False) - B 28 06:46:14.901 PM OnApplicationPause(True) - B 29 06:46:14.128 PM SendBusinessDataToFireStore - BOARD 30 06:46:12.682 PM SendUserDataToFireStore - B 31 06:46:12.681 PM OnApplicationFocus(False) - B 32 06:46:10.828 PM SendBusinessDataToFireStore - BOARD 33 06:46:10.592 PM SendUserDataToFireStore - B 34 06:46:10.591 PM OnApplicationFocus(False) - B 35 06:45:54.736 PM SendBusinessDataToFireStore - BOARD 36 06:45:54.588 PM SendUserDataToFireStore - B 37 06:45:46.441 PM SendBusinessDataToFireStore - BOARD 38 06:45:46.277 PM SendUserDataToFireStore - B 39 06:45:30.919 PM SendBusinessDataToFireStore - BOARD 40 06:45:30.770 PM SendUserDataToFireStore - B 41 06:45:28.263 PM SetItemMiniTutorialState boardIdx 18 itemID 411 Cleaning Bucket 42 06:45:25.183 PM roadmap_free_reward 43 06:45:21.997 PM SendBusinessDataToFireStore - BOARD 44 06:45:21.799 PM SendUserDataToFireStore - B 45 06:45:13.307 PM SendBusinessDataToFireStore - BOARD 46 06:45:13.050 PM SendUserDataToFireStore - B 47 06:45:01.284 PM SendBusinessDataToFireStore - BOARD 48 06:44:59.604 PM SendUserDataToFireStore - B 49 06:44:52.892 PM SendBusinessDataToFireStore - BOARD 50 06:44:52.644 PM SendUserDataToFireStore - B 51 06:44:37.534 PM SendBusinessDataToFireStore - BOARD 52 06:44:37.402 PM SendUserDataToFireStore - B 53 06:44:34.879 PM SendAnalytics to collectionAnalyticsRoadmapB 54 06:44:34.836 PM SendBusinessDataToFireStore - BOARD 55 06:44:34.670 PM SendUserDataToFireStore - B 56 06:44:24.975 PM SendBusinessDataToFireStore - BOARD 57 06:44:24.810 PM SendUserDataToFireStore - B 58 06:44:21.327 PM SendBusinessDataToFireStore - BOARD 59 06:44:21.129 PM SendUserDataToFireStore - B 60 06:44:03.905 PM SendBusinessDataToFireStore - BOARD 61 06:44:03.723 PM SendUserDataToFireStore - B 62 06:44:01.306 PM SendBusinessDataToFireStore - BOARD 63 06:44:01.158 PM SendUserDataToFireStore - B 64 06:43:14.283 PM SendBusinessDataToFireStore - BOARD 65 06:43:14.101 PM SendUserDataToFireStore - B 66 06:43:14.100 PM OnApplicationFocus(False) - B 67 06:43:04.474 PM SendBusinessDataToFireStore - BOARD 68 06:43:04.342 PM SendUserDataToFireStore - B 69 06:43:01.126 PM SendBusinessDataToFireStore - BOARD 70 06:43:00.911 PM SendUserDataToFireStore - B 71 06:42:51.334 PM SendBusinessDataToFireStore - BOARD 72 06:42:51.168 PM SendUserDataToFireStore - B 73 06:42:33.848 PM SendBusinessDataToFireStore - BOARD 74 06:42:33.667 PM SendUserDataToFireStore - B 75 06:42:20.307 PM SendBusinessDataToFireStore - BOARD 76 06:42:20.142 PM SendUserDataToFireStore - B 77 06:42:08.237 PM SendBusinessDataToFireStore - META 78 06:42:08.039 PM SendUserDataToFireStore - B 79 06:41:44.232 PM SendBusinessDataToFireStore - BOARD 80 06:41:44.084 PM SendUserDataToFireStore - B 81 06:41:39.877 PM SendBusinessDataToFireStore - BOARD 82 06:41:39.729 PM SendUserDataToFireStore - B 83 06:41:11.505 PM roadmap_free_reward 84 06:41:04.819 PM SendBusinessDataToFireStore - BOARD 85 06:41:04.671 PM SendUserDataToFireStore - B 86 06:41:02.148 PM SendAnalytics to collectionAnalyticsRoadmapB 87 06:40:50.762 PM SendBusinessDataToFireStore - BOARD 88 06:40:50.564 PM SendUserDataToFireStore - B 89 06:40:38.258 PM SendBusinessDataToFireStore - BOARD 90 06:40:38.083 PM SendUserDataToFireStore - B 91 06:40:29.532 PM SendBusinessDataToFireStore - BOARD 92 06:40:29.392 PM SendUserDataToFireStore - B 93 06:40:23.386 PM SendBusinessDataToFireStore - BOARD 94 06:40:23.188 PM SendUserDataToFireStore - B 95 06:40:09.006 PM SendBusinessDataToFireStore - BOARD 96 06:40:08.808 PM SendUserDataToFireStore - B 97 06:40:03.885 PM SendBusinessDataToFireStore - BOARD 98 06:40:03.670 PM SendUserDataToFireStore - B 99 06:40:01.157 PM soft_tutorial_stepB 100 06:40:01.137 PM SendAnalytics to collectionAnalyticsFTUESoftB 101 06:40:00.912 PM soft_tutorial_stepB 102 06:40:00.888 PM SendAnalytics to collectionAnalyticsFTUESoftB 103 06:39:58.461 PM soft_tutorial_stepB 104 06:39:58.448 PM SendAnalytics to collectionAnalyticsFTUESoftB 105 06:39:57.848 PM SendBusinessDataToFireStore - META 106 06:39:57.700 PM SendUserDataToFireStore - B 107 06:39:55.196 PM soft_tutorial_stepB 108 06:39:55.185 PM SendAnalytics to collectionAnalyticsFTUESoftB 109 06:39:54.307 PM soft_tutorial_stepB 110 06:39:54.293 PM SendAnalytics to collectionAnalyticsFTUESoftB 111 06:39:51.893 PM soft_tutorial_stepB 112 06:39:51.878 PM SendAnalytics to collectionAnalyticsFTUESoftB 113 06:39:49.879 PM SendBusinessDataToFireStore - BOARD 114 06:39:49.747 PM SendUserDataToFireStore - B 115 06:39:46.148 PM SendBusinessDataToFireStore - BOARD 116 06:39:45.983 PM SendUserDataToFireStore - B 117 06:39:37.304 PM SendBusinessDataToFireStore - BOARD 118 06:39:37.148 PM SendUserDataToFireStore - B 119 06:39:37.147 PM OnApplicationFocus(False) - B 120 06:39:36.597 PM soft_tutorial_stepB 121 06:39:36.589 PM SendAnalytics to collectionAnalyticsFTUESoftB 122 06:39:35.220 PM soft_tutorial_stepB 123 06:39:35.207 PM SendAnalytics to collectionAnalyticsFTUESoftB 124 06:39:35.015 PM soft_tutorial_stepB 125 06:39:35.007 PM SendAnalytics to collectionAnalyticsFTUESoftB 126 06:39:33.242 PM SendBusinessDataToFireStore - BOARD 127 06:39:33.110 PM SendUserDataToFireStore - B 128 06:39:28.946 PM soft_tutorial_stepB 129 06:39:28.940 PM roadmap_free_reward 130 06:39:28.931 PM SendAnalytics to collectionAnalyticsFTUESoftB 131 06:39:27.591 PM SendBusinessDataToFireStore - ROADMAP 132 06:39:27.388 PM SendUserDataToFireStore - B 133 06:39:24.714 PM soft_tutorial_stepB 134 06:39:24.705 PM SendAnalytics to collectionAnalyticsFTUESoftB 135 06:39:24.273 PM SendBusinessDataToFireStore - BOARD 136 06:39:24.072 PM SendUserDataToFireStore - B 137 06:39:21.572 PM soft_tutorial_stepB 138 06:39:21.563 PM SendAnalytics to collectionAnalyticsRoadmapB 139 06:39:21.557 PM SendAnalytics to collectionAnalyticsFTUESoftB 140 06:39:21.032 PM SendBusinessDataToFireStore - BOARD 141 06:39:20.800 PM SendUserDataToFireStore - B 142 06:39:10.895 PM SendBusinessDataToFireStore - BOARD 143 06:39:10.598 PM SendUserDataToFireStore - B 144 06:39:01.600 PM SendBusinessDataToFireStore - BOARD 145 06:39:01.402 PM SendUserDataToFireStore - B 146 06:39:01.397 PM soft_tutorial_stepB 147 06:39:01.385 PM SendAnalytics to collectionAnalyticsFTUESoftB 148 06:38:59.413 PM soft_tutorial_stepB 149 06:38:59.402 PM SendAnalytics to collectionAnalyticsFTUESoftB 150 06:38:59.151 PM soft_tutorial_stepB 151 06:38:59.136 PM SendAnalytics to collectionAnalyticsFTUESoftB 152 06:38:58.901 PM soft_tutorial_stepB 153 06:38:58.887 PM SendAnalytics to collectionAnalyticsFTUESoftB 154 06:38:57.836 PM SendBusinessDataToFireStore - BOARD 155 06:38:57.621 PM SendUserDataToFireStore - B 156 06:38:55.110 PM soft_tutorial_stepB 157 06:38:55.097 PM SendAnalytics to collectionAnalyticsFTUESoftB 158 06:38:53.723 PM SendBusinessDataToFireStore - META 159 06:38:53.499 PM SendUserDataToFireStore - B 160 06:38:50.995 PM soft_tutorial_stepB 161 06:38:50.984 PM SendAnalytics to collectionAnalyticsFTUESoftB 162 06:38:50.232 PM soft_tutorial_stepB 163 06:38:50.218 PM SendAnalytics to collectionAnalyticsFTUESoftB 164 06:38:48.016 PM soft_tutorial_stepB 165 06:38:48.003 PM SendAnalytics to collectionAnalyticsFTUESoftB 166 06:38:44.869 PM soft_tutorial_stepB 167 06:38:44.857 PM SendAnalytics to collectionAnalyticsFTUESoftB 168 06:38:43.707 PM soft_tutorial_stepB 169 06:38:43.693 PM SendAnalytics to collectionAnalyticsFTUESoftB 170 06:38:43.457 PM soft_tutorial_stepB 171 06:38:43.443 PM SendAnalytics to collectionAnalyticsFTUESoftB 172 06:38:41.095 PM soft_tutorial_stepB 173 06:38:41.080 PM SendAnalytics to collectionAnalyticsFTUESoftB 174 06:38:39.132 PM soft_tutorial_stepB 175 06:38:39.124 PM roadmap_free_reward 176 06:38:39.115 PM SendAnalytics to collectionAnalyticsFTUESoftB 177 06:38:37.424 PM SendBusinessDataToFireStore - ROADMAP 178 06:38:37.255 PM SendUserDataToFireStore - B 179 06:38:34.579 PM soft_tutorial_stepB 180 06:38:34.565 PM SendAnalytics to collectionAnalyticsFTUESoftB 181 06:38:31.682 PM SendBusinessDataToFireStore - BOARD 182 06:38:31.485 PM SendUserDataToFireStore - B 183 06:38:28.979 PM soft_tutorial_stepB 184 06:38:28.966 PM SendAnalytics to collectionAnalyticsRoadmapB 185 06:38:28.960 PM SendAnalytics to collectionAnalyticsFTUESoftB 186 06:38:19.590 PM SendBusinessDataToFireStore - BOARD 187 06:38:19.425 PM SendUserDataToFireStore - B 188 06:38:17.172 PM soft_tutorial_stepB 189 06:38:17.159 PM SendAnalytics to collectionAnalyticsFTUESoftB 190 06:38:16.924 PM soft_tutorial_stepB 191 06:38:16.908 PM SendAnalytics to collectionAnalyticsFTUESoftB 192 06:38:09.620 PM SendBusinessDataToFireStore - BOARD 193 06:38:09.406 PM SendUserDataToFireStore - B 194 06:38:05.390 PM SendBusinessDataToFireStore - BOARD 195 06:38:04.857 PM SendUserDataToFireStore - B 196 06:37:58.199 PM soft_tutorial_stepB 197 06:37:58.179 PM SendAnalytics to collectionAnalyticsFTUESoftB 198 06:37:57.946 PM soft_tutorial_stepB 199 06:37:57.929 PM SendAnalytics to collectionAnalyticsFTUESoftB 200 06:37:50.451 PM soft_tutorial_stepB 201 06:37:50.437 PM SendAnalytics to collectionAnalyticsFTUESoftB 202 06:37:48.621 PM soft_tutorial_stepB 203 06:37:48.598 PM SendAnalytics to collectionAnalyticsFTUESoftB 204 06:37:48.383 PM soft_tutorial_stepB 205 06:37:48.372 PM tutorial_completeB 206 06:37:48.358 PM tutorial_step 207 06:37:46.298 PM tutorial_step 208 06:37:44.284 PM tutorial_step 209 06:37:42.918 PM tutorial_step 210 06:37:42.104 PM tutorial_step 211 06:37:41.187 PM tutorial_step 212 06:37:40.221 PM tutorial_step 213 06:37:38.737 PM tutorial_step 214 06:37:37.510 PM SendBusinessDataToFireStore - TUTORIAL_BOARD 215 06:37:37.073 PM tutorial_step 216 06:37:35.993 PM tutorial_step 217 06:37:35.413 PM SendUserDataToFireStore - B 218 06:37:32.912 PM tutorial_step 219 06:37:31.901 PM tutorial_step 220 06:37:30.195 PM tutorial_step 221 06:37:28.564 PM SendBusinessDataToFireStore - TUTORIAL_META 222 06:37:28.333 PM SendUserDataToFireStore - B 223 06:37:25.837 PM tutorial_step 224 06:37:05.994 PM gpdr_accepted 225 06:37:05.986 PM UserID ztm1OYyRbqdqE51IdIDOXVpyvla2 - B 226 06:37:05.955 PM tutorial_start 227 06:37:05.951 PM SendAnalytics to collectionAnalyticsRetentionB
Yep, those objects look like heavy global ref consumers. Probably what is happening is that the Firestore C# objects are accumulating in memory, even though they are no longer referenced. Eventually, the garbage collector runs, destroys the C# objects, and those objects' finalizers are run to free the Android global refs. But if the number of global refs hits the 51200 hard maximum before the garbage collector runs then the exhaustion occurs, resulting in the crash. That's probably why it takes 20 minutes or so before the global refs accumulate to a problematic level.
The only workaround that I can think of would be to call FirebaseApp.Dispose()
periodically to free some global refs. This is, of course, a very heavy operation and you will need to re-create the FirebaseApp and the FirebaseFirestore, etc. again from scratch. But I'm throwing it out there as a potential workaround.
Is Firebase using its own garbage collector or is it using Unity's?
Can I manually cache and re-use the document refs instead of creating new ones?
Firebase is not using its own garbage collector; it's using Unity's.
No, there is no way to manually re-use the document refs instead of creating new ones?
In that case, would calling System.GC.Collect() or Resources.UnloadUnusedAssets manually at opportune times solve the issue?
It's worth a try, although I don't know for sure. It depends how the garbage collector responds to those calls.
FYI, unfortunately calling: Resources.UnloadUnusedAssets(); GC.Collect(); Does not solve the problem.
The only solution we found is:
At least on some devices, there doesn't seem to be any clearing of the reference pool, ever, during gameplay. Possibly on all devices.
To me it looks more and more like you guys have a memory leak in the code.
Interesting. It's entirely possible that there is a memory leak in Firestore. Would you be able to use something like Android's "Memory Profiler" to see if you can get some more details about the memory usage of your app, and/or find some evidence of the unbounded memory growth? This information could point to the part of the Firestore SDK that is leaking memory.
https://developer.android.com/studio/profile/memory-profiler
Hi, I do not see any sort of leak in the memory profiler. That said, I don't think we would see one. The issue is not new memory being allocated, its that the reference pool is not clearing itself correctly when the docRef is garbage collected. I assume the pool would be pre-allocated to 51200 elements, so adding to the pool wouldn't show up in the memory profiler anyways.
Okay. I was hoping the memory profiler would give some useful information. Thanks for trying it out. You may be right that there is a leak somewhere in Firestore, but for now we will focus our attention on fixing the exhaustion of the Android global reference pool. The fix is still expected in the next release.
Hi, another question. You mentioned an array of X elements would be X+1 references (if I understood you correctly.) What about a string? Would a string be 1 reference or would it be treated like an array of bytes?
A String is just 1 global reference (the reference to the java.lang.String Java object).
Hi, About 15% of our users seem to have Firebase related crashes, it seems when saving to FireStore.
After uploading symbols to Google Play, we managed to get the following crash data in the Google Play Console:
[libFirebaseCppApp-10_1_1.so] firebase::firestore::FieldValueInternal::FieldValueInternal(long)
[libFirebaseCppApp-10_1_1.so] firebase::firestore::jni::Global<firebase::firestore::jni::Object>::Global(firebase::firestore::jni::Object const&)
[libFirebaseCppApp-10_1_1.so] firebase::firestore::FieldValue::FieldValue(firebase::firestore::FieldValue const&)
Some additional information: After adding symbols to Firebase, we have the following crashes on crashlytics:
All three crashes are happening when saving our UserData to FireStore, and they are each happening to different users. So a user who experiences one does not experience the others.
Any ideas?
Thanks