io.realm.exceptions.RealmFileException when launching application

jollyjoker992 commented 7 years ago

I have researched many and many times about this issue but I have not found any solution. I am implementing an Android application with Realm Android 3.0 and using Fabric to find out crash report. Below is Realm initialize code in Application class:

 Realm.init(this);
        RealmConfiguration config = new RealmConfiguration.Builder().name(REALM_SCHEMA_NAME)
                .schemaVersion(REALM_SCHEMA_VERSION)
                .migration(new Migration())
                .build();
        Realm.setDefaultConfiguration(config);
        Realm realm = Realm.getDefaultInstance(); // Automatically run migration if needed
        realm.close();

I get many crash report from Fabric related with io.realm.exceptions.RealmFileException such as:

Unable to open a realm at path '/data/data/com.xxx/files/xxx.realm': Incompatible histories.
Unable to open a realm at path '/data/data/com.xxx/files/xxx.realm': bad realm file header (#3). (bad realm file header (#3))
Unrecoverable error. mmap() failed: Out of memory size xxx
etc ...

So, can you tell me the reason of this issue? Thank you.

Zhuinden commented 7 years ago

mmap() failed is generally caused by unclosed Realm instances on background threads.

Bad file header can be caused by compacting in multiple processes.

jollyjoker992 commented 7 years ago

@Zhuinden : I do not use compact method, so why do it happen?

kneth commented 7 years ago

@jollyjoker992 Does your app have multiple processes? Does it happen the first time your app is launched? Did the device run out of disk space?

jollyjoker992 commented 7 years ago

@kneth : No, my app is single process. It does not happen in the first time launching app, It happens in many time when launching app. I have attached code above and it is located in Application class. From Fabric report, all devices has enough free disk space.

kneth commented 7 years ago

@jollyjoker992 Is there a pattern in which devices you see the crashes? How easy is it to reproduce (i.e., how often does it happen)?

jollyjoker992 commented 7 years ago

@kneth : Because I get crash report from users by Fabric, so I cannot reproduce it. So, sometime It happens when I launch app, I trace log and see all of issues I mentioned above. From Fabric, it happens in:

Devices : SHAFT (41%), Sony (34%), Fujisu (24%) and all of them have enough free disk space and do not root before.
OS : 6 (76%), 5 (13%), 7 (6%)

Can you tell me all of reasons make all of above issue? I will try to verify each reason and hope find out the root cause. Thank you.

kneth commented 7 years ago

@jollyjoker992 So no obvious patterns 😢

If you can post the stack trace/log from one or two of the crashes, it might be useful.

jollyjoker992 commented 7 years ago

@kneth : Below is one of crash logs I just found right now. I will update more log later. Thank you.

Fatal Exception: java.lang.RuntimeException: Unable to create application com.xxx.xxx.xxxApplication: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.xxx.xxx.xxx/files/xxx.realm': Bad Realm file header (#3). (Bad Realm file header (#3)) (/data/data/com.xxx.xxx.xxx/files/xxx.realm) in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 109 Kind: ACCESS_ERROR.
       at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4809)
       at android.app.ActivityThread.access$1600(ActivityThread.java:154)
       at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1452)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:224)
       at android.app.ActivityThread.main(ActivityThread.java:5526)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
Caused by io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.xxx.xxx.xxx/files/xxx.realm': Bad Realm file header (#3). (Bad Realm file header (#3)) (/data/data/com.xxx.xxx.xxx/files/xxx.realm) in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 109
       at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
       at io.realm.internal.SharedRealm.<init>(SharedRealm.java:187)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:229)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:204)
       at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:124)
       at io.realm.Realm.getDefaultInstance(Realm.java:210)
       at com.lab.xxx.xxxApplication.initAndMigrateRealmIfNeeded(xxxApplication.java:60)
       at com.lab.xxx.xxxApplication.onCreate(xxxApplication.java:45)
       at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1014)
       at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4806)
       at android.app.ActivityThread.access$1600(ActivityThread.java:154)
       at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1452)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:224)
       at android.app.ActivityThread.main(ActivityThread.java:5526)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)

Zhuinden commented 7 years ago

A possible fix for I think this error is being investigated in https://github.com/realm/realm-java/pull/4559

kneth commented 7 years ago

@jollyjoker992 With #4559 merged, you can try a snapshot release and see if it solves your issue.

jollyjoker992 commented 7 years ago

@Zhuinden @kneth Thanks for your support. I will try to use snapshot. But, can you tell me about plan to release version includes this commit?

kneth commented 7 years ago

@jollyjoker992 No exact date but we will probably do a release in a week or two (we don't have fixed date as support load or critical bugs are hard to predict).

jollyjoker992 commented 7 years ago

@kneth : I tried to use Snap shot for testing this issue. So, Can you close this issue? I will re-open it or note for you after a few days for testing. Thank you for your support.

jollyjoker992 commented 7 years ago

@kneth : Hi. I have tried to use Realm Snap-shot 3.2.0 to test the latest version (the last version is 3.0.0). The issue 'Bad realm file header' maybe resolved, I have not seen it in crash report. But, the below issues still occurs.

Unable to open a realm at path '/data/data/com.xxx/files/xxx.realm': Incompatible histories.
Unrecoverable error. mmap() failed: Out of memory size xxx

I close Realm instance follow App life cycle, but when system kills my app (on low memory case), I cannot close it. So, Is it a root cause? Pls help me to investigate problem by showing me some cases that these issues may occur. Thank you so much.

Zhuinden commented 7 years ago

Unrecoverable error. mmap() failed: Out of memory size xxx

This is generally caused if the Realm file is too large, which is generally when there is at least one unclosed Realm instance on a non-looper background thread.

jollyjoker992 commented 7 years ago

@Zhuinden : Thank you for your reply. As I mentioned above:

I close Realm instance follow App life cycle, but when system kills my app (on low memory case), I cannot close it.

Is it a root cause of this issue?

Unrecoverable error. mmap() failed: Out of memory size xxx

Zhuinden commented 7 years ago

No, Realm opened on looper thread doesn't create a lot of overhead, although if you never close it then I wonder how you compact.

jollyjoker992 commented 7 years ago

@Zhuinden : Thank you. I got it. But, I got crash report related below issue just now.

Fatal Exception: java.lang.RuntimeException: Unable to create application com.xxx.xxx.xxxApplication: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.xxx.xxx.xxx/files/xxx.realm': Bad Realm file header (#3). (Bad Realm file header (#3)) (/data/data/com.xxx.xxx.xxx/files/xxx.realm) in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 109 Kind: ACCESS_ERROR. at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4809) at android.app.ActivityThread.access$1600(ActivityThread.java:154) at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1452) at android.os.Handler.dispatchMessage(Handler.java:102) at android.os.Looper.loop(Looper.java:224) at android.app.ActivityThread.main(ActivityThread.java:5526) at java.lang.reflect.Method.invoke(Method.java) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616) Caused by io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.xxx.xxx.xxx/files/xxx.realm': Bad Realm file header (#3). (Bad Realm file header (#3)) (/data/data/com.xxx.xxx.xxx/files/xxx.realm) in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 109 at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java) at io.realm.internal.SharedRealm.(SharedRealm.java:187) at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:229) at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:204) at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:124) at io.realm.Realm.getDefaultInstance(Realm.java:210) at com.lab.xxx.xxxApplication.initAndMigrateRealmIfNeeded(xxxApplication.java:60) at com.lab.xxx.xxxApplication.onCreate(xxxApplication.java:45) at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1014) at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4806) at android.app.ActivityThread.access$1600(ActivityThread.java:154) at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1452) at android.os.Handler.dispatchMessage(Handler.java:102) at android.os.Looper.loop(Looper.java:224) at android.app.ActivityThread.main(ActivityThread.java:5526) at java.lang.reflect.Method.invoke(Method.java) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)

It seems SNAP-SHOT 3.2.0 version cannot help me. So boring about that :(

jollyjoker992 commented 7 years ago

@Zhuinden : Does Realm compact method help us to avoid realm file size increase when we cannot close Realm in a looper thread (Ex: System kill app)? Thank you

kneth commented 7 years ago

@jollyjoker992 Compacting a Realm will removed other versions and other unused space in the Realm file. Most databases have such a function (in PostgreSQL it is called VACUUM). We don't support online compaction so doing it a launch after the app has been killed sounds like a good time to do it.

jollyjoker992 commented 7 years ago

I have upgraded to realm 3.1.4 but this bugs still happen. The latest log I get from Crash Report is below:

Fatal Exception: java.lang.RuntimeException: Unable to create application com.xxx: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.xxx/files/xxx.realm': Incompatible histories. Expected a Realm with no or in-realm history. (Incompatible histories. Expected a Realm with no or in-realm history) (/data/data/com.xxx/files/xxx.realm) in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 217 Kind: ACCESS_ERROR.
       at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5494)
       at android.app.ActivityThread.-wrap2(ActivityThread.java)
       at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1583)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:241)
       at android.app.ActivityThread.main(ActivityThread.java:6217)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:865)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:755)
Caused by io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.xxx/files/xxx.realm': Incompatible histories. Expected a Realm with no or in-realm history. (Incompatible histories. Expected a Realm with no or in-realm history) (/data/data/com.xxx/files/xxx.realm) in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 217
       at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
       at io.realm.internal.SharedRealm.<init>(SharedRealm.java:190)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:237)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:206)
       at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:127)
       at io.realm.Realm.getDefaultInstance(Realm.java:266)
       at com.lab.xxx.data.source.local.RealmProcessor.init(RealmProcessor.java:29)
       at com.lab.xxx.MapionApplication.onCreate(MapionApplication.java:30)
       at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1025)
       at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5491)
       at android.app.ActivityThread.-wrap2(ActivityThread.java)
       at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1583)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:241)
       at android.app.ActivityThread.main(ActivityThread.java:6217)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:865)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:755)

Sometime is Bad Realm File Header, sometime is Incompatible histories. It happens about ~ 200 times and affect to ~20 users in 2 weeks. So, I think it is very important.

Pls help me to investigate it. Thank you!

kneth commented 7 years ago

@jollyjoker992 Does it happen when the app is updated?

Zhuinden commented 7 years ago

I can't help but feel like calling Realm.getDefaultInstance() inside Application.onCreate() is what can cause this kind of thing.

This is why I moved Realm lifecycle counting to Activities instead.

jollyjoker992 commented 7 years ago

@kneth : No, It had happened before I upgraded to version 3.1.4. You can find all comments I mentioned above. I could not find the root cause. I attached my code in previous comments. Pls help me to investigate this issue. Thank you!

jollyjoker992 commented 7 years ago

@Zhuinden : I called it in Application onCreate because of migration. Is that incorrect?

Zhuinden commented 7 years ago

If you have multiple processes in the app, then if you do it in application.onCreate(), then both processes will start a migration, and they'll break the Realm file into unrecoverable pieces.

jollyjoker992 commented 7 years ago

@Zhuinden : As I mentioned above, my app uses a local service and running in the same process with application process. So, Is that a problem?

kneth commented 7 years ago

@jollyjoker992 My question wasn't about upgrading Realm but the app upgrade procedure. We have for a while had a hypothesis that the android installer might cause issues when you have open files.

jollyjoker992 commented 7 years ago

@kneth : I understand what you say but if It's the problem we are finding out, How can I resolve it ?

jollyjoker992 commented 7 years ago

@kneth @Zhuinden : I have seen from other issues that relates my issue. I know that currently you guess that the root cause may be new app version upgrade and the app runs on multi process. My app runs on single process, so I think It is not a root cause. Please help me to resolve the problem. Thank you.

kneth commented 7 years ago

@jollyjoker992 In order to solve it, we need to be able to reproduce it. Since your app and reproduce it, you are probably in the best one to come up with the conditions/procedure. Is it possible for you to share your source code with us? You don't have to do it in public and reveal all your secrets :smile:

jollyjoker992 commented 7 years ago

@kneth : Sorry because I cannot. My project has private rule and I cannot share code with you. But, I can share how I manage Realm in my project. Can you support online? Because of different timezone (I am GMT +7 and I guess you are in Europe time zone), so I have to wait about 1 day to get feedback from you :( So, Can you help me to support online via skype or another tool ? Thank you so much.

kneth commented 7 years ago

@jollyjoker992 We only do skype support for paying customers. Please contact sales@realm.io if you are interested in learning more about our troubleshooting packages.

Zhuinden commented 7 years ago

@kneth i didn't know you guys had Skype video chat support things

jollyjoker992 commented 6 years ago

@kneth @Zhuinden : I do not know that have you fixed this bug in the latest version of Realm? It still happens in our app and the affected users is increase faster at the latest release version (Of course, we do not change anything relates to Realm process). Thank you.

kneth commented 6 years ago

@jollyjoker992 A number of fixes has been going into core recently so upgrading might be a good idea.

jollyjoker992 commented 6 years ago

@kneth : I have upgraded Realm to version 4.0.0-RC1 but this issues still occurred many times. So, which version you mentioned above might fix these issues? Thank you.

beeender commented 6 years ago

@jollyjoker992

Please notice that exception caused by Unable to open a realm xxx probably means the db file is corrupted for some reasons. You also need to delete the Realm file (or use a new Realm file with a different name) as well as using the new realm-java version. Otherwise the corrupted file will still be corrupted.
Do you have the full trace of Unable to open a realm at path '/data/data/com.xxx/files/xxx.realm': Incompatible histories. ?
Full trace of Unrecoverable error. mmap() failed: Out of memory size xxx ?
When Unrecoverable error. mmap() failed happens, did you ever cat the exception and continue? If yes, there would be a big chance to corrupt the realm file.

jollyjoker992 commented 6 years ago

@beeender : Thank for your response.

Can I catch this Exception in Java code? When I delete the Realm file (or use a new Realm file with a different name), Can I keep (or restore) data?

Fatal Exception: java.lang.RuntimeException: Unable to create application com.lab.xxx.xxxApplication: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.xxx.android.arukuto/files/xxx.realm': Incompatible histories. Expected a Realm with no or in-realm history. (Incompatible histories. Expected a Realm with no or in-realm history) (/data/data/com.xxx.android.arukuto/files/xxx.realm) in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 101 Kind: ACCESS_ERROR.
   at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5573)
   at android.app.ActivityThread.-wrap3(ActivityThread.java)
   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1621)
   at android.os.Handler.dispatchMessage(Handler.java:102)
   at android.os.Looper.loop(Looper.java:154)
   at android.app.ActivityThread.main(ActivityThread.java:6286)
   at java.lang.reflect.Method.invoke(Method.java)
   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:869)
   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:759)
Caused by io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.xxx.android.arukuto/files/xxx.realm': Incompatible histories. Expected a Realm with no or in-realm history. (Incompatible histories. Expected a Realm with no or in-realm history) (/data/data/com.xxx.android.arukuto/files/xxx.realm) in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 101
   at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
   at io.realm.internal.SharedRealm.<init>(SharedRealm.java:194)
   at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:241)
   at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:231)
   at io.realm.BaseRealm.compactRealm(BaseRealm.java:636)
   at io.realm.Realm.compactRealm(Realm.java:1685)
   at com.lab.xxx.data.source.local.RealmProcessor.compact(RealmProcessor.java:34)
   at com.lab.xxx.data.source.local.RealmProcessor.init(RealmProcessor.java:29)
   at com.lab.xxx.xxxApplication.onCreate(xxxApplication.java:41)
   at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1025)
   at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5570)
   at android.app.ActivityThread.-wrap3(ActivityThread.java)
   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1621)
   at android.os.Handler.dispatchMessage(Handler.java:102)
   at android.os.Looper.loop(Looper.java:154)
   at android.app.ActivityThread.main(ActivityThread.java:6286)
   at java.lang.reflect.Method.invoke(Method.java)
   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:869)
   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:759)

Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. mmap() failed: Out of memory size: 1207959552 offset: 0 in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 217
   at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
   at io.realm.internal.SharedRealm.<init>(SharedRealm.java:190)
   at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:237)
   at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:206)
   at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:127)
   at io.realm.Realm.getDefaultInstance(Realm.java:266)
   at com.xxx.xxx.data.source.local.RealmProcessor.init(RealmProcessor.java:29)
   at com.xxx.xxx.xxxApplication.onCreate(xxxApplication.java:41)
   at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1014)
   at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4804)
   at android.app.ActivityThread.access$1600(ActivityThread.java:154)
   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1452)
   at android.os.Handler.dispatchMessage(Handler.java:102)
   at android.os.Looper.loop(Looper.java:234)
   at android.app.ActivityThread.main(ActivityThread.java:5524)
   at java.lang.reflect.Method.invoke(Method.java)
   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)

No, Pls explain me how to catch it in Java code?

Other log:

Fatal Exception: java.lang.RuntimeException: Unable to create application com.lab.xxx.xxxApplication: io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.xxx.android.arukuto/files/xxx.realm': Bad Realm file header (#1). (Bad Realm file header (#1)) (/data/data/com.xxx.android.arukuto/files/xxx.realm) in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 217 Kind: ACCESS_ERROR.
   at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4807)
   at android.app.ActivityThread.access$1600(ActivityThread.java:154)
   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1452)
   at android.os.Handler.dispatchMessage(Handler.java:102)
   at android.os.Looper.loop(Looper.java:234)
   at android.app.ActivityThread.main(ActivityThread.java:5524)
   at java.lang.reflect.Method.invoke(Method.java)
   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
Caused by io.realm.exceptions.RealmFileException: Unable to open a realm at path '/data/data/com.xxx.android.arukuto/files/xxx.realm': Bad Realm file header (#1). (Bad Realm file header (#1)) (/data/data/com.xxx.android.arukuto/files/xxx.realm) in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 217
   at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
   at io.realm.internal.SharedRealm.<init>(SharedRealm.java:190)
   at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:237)
   at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:206)
   at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:127)
   at io.realm.Realm.getDefaultInstance(Realm.java:266)
   at com.lab.xxx.data.source.local.RealmProcessor.init(RealmProcessor.java:29)
   at com.lab.xxx.xxxApplication.onCreate(xxxApplication.java:41)
   at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1014)
   at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4804)
   at android.app.ActivityThread.access$1600(ActivityThread.java:154)
   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1452)
   at android.os.Handler.dispatchMessage(Handler.java:102)
   at android.os.Looper.loop(Looper.java:234)
   at android.app.ActivityThread.main(ActivityThread.java:5524)
   at java.lang.reflect.Method.invoke(Method.java)
   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)

beeender commented 6 years ago

1.

Can I catch this Exception in Java code? When I delete the Realm file (or use a new Realm file with a different name), Can I keep (or restore) data?

Normally the corrupted db should never happen ... but if it happened before, the db file has been corrupted already, you can catch the exception and delete the Realm file. But please notice, you should only catch it at the first time Realm file was opened (like in Application.onCreate() )

from the mmap exception you posted, I think you still have problems which cause the db file grows unexpected. I believe there are still some Realm instances on the background threads are not closed properly. If the db size is too big, and you are calling the compactRealm(), the db file could be corrupted in the earlier version, which has been fixed by https://github.com/realm/realm-core/pull/2852. So please check your Realm usage on the background thread again. some APIs might be helpful : https://realm.io/docs/java/4.1.0/api/io/realm/Realm.html#getLocalInstanceCount-io.realm.RealmConfiguration- and https://realm.io/docs/java/4.1.0/api/io/realm/Realm.html#getGlobalInstanceCount-io.realm.RealmConfiguration-
No, Pls explain me how to catch it in Java code?

Something like in your Applicaton.onCreate():

try {
Realm realm = Realm.getInstance(config);
} catch (RealmFileException e) {
    if (e.getMessage().contains("Incompatible histories. Expected a Realm with no or in-realm history.") {
        Realm.deleteRealm(config);
        // Start again
    } else {
        // should not happen, but don't swallow the exception silently.
        throw e;
    }
}

Again, this should only be called at the first time you open the Realm instance. And please use realm-java v4.1.0 which has process-safety guarantee for the Realm.deleteRealm() just in case you have multi-processes in your application.

jollyjoker992 commented 6 years ago

@beeender : Thanks for your response.

Can I keep (or restore) data?

Pls help me to restore data after I deleted it. I want to keep all data in previous Realm file and users do not know about delete action.

I use only one instance of Realm in Main Thread and do not use any Realm instance in another background thread.

beeender commented 6 years ago

Can I keep (or restore) data?

Sorry, if the db is corrupted, the data will be lost unfortunately.

I use only one instance of Realm in Main Thread and do not use any Realm instance in another background thread.

That would be very strange! unless you really write a lot of data into Realm, the db file shouldn't be that big. Can you share your apk with us so i can help to take a look? you can send it to help@realm.io if you want to share it privately.

Zhuinden commented 6 years ago

Are you sure that Realm.getGlobalInstanceCount(config) is always 1? And never 2 or higher?

jollyjoker992 commented 6 years ago

@beeender @Zhuinden Currently I am waiting the approval to share APK for you. So, I have a question. When I catch the Realm file is corrupted, I want to delete and re-create it, but before delete it, I must close all Realm instance. So, If having another Realm instance in another background thread (For example, when I do more than 1 async transaction, multi background Realm instance is created and still open util the transaction is complete), so I cannot delete Realm file. So, pls show me the way to close all Realm instances that still exists and has not closed yet before delete Realm file. Thank you.

cmelchior commented 6 years ago

If your Realm file is corrupted, it shouldn't be possible to open it on any thread?

jollyjoker992 commented 6 years ago

@cmelchior : I think there are some cases meets my above question. For example, when I open more than 2 async transaction at the same time, we have more than 2 Realm instance in back ground thread but one of them throws the corrupted exception, another instance is in non-finished transaction and when I catch this exception, I delete the realm file immediately but I cannot because there are a realm instance has not been closed :(

jollyjoker992 commented 6 years ago

@cmelchior @Zhuinden @kneth Thanks for your support. I'm currently catching the Exception when openning Realm instance then delete it if need and recover Realm file by supporting from server side.

Cannot delete Realm file although I use only 1 instance in Main thread but when I put the log to get global Realm instance count, I saw that there are 2 instances and 1 instance randomly. I guess that there are a transaction async is in progress, so I block this thread until the getGlobalInstanceCount return 1. It's resolved.
I will try to reduce opening many transactions at the same time and hope it will be resolved but I have a question for you.

If open many transactions at the same time causes Realm file size becomes too large, so It seems like a bug, right? Can you control it to reduce the Realm file size like compact method is defined ?

Zhuinden commented 6 years ago

@jollyjoker992 have you tried with Realm 4.2.0? It should have a bunch of multi-process related fixes (because of Realm Core 4.0.3).

jollyjoker992 commented 6 years ago

@Zhuinden : No, I have not yet but I will try it :+1: Thanks you

jollyjoker992 commented 6 years ago

@Zhuinden : Realm 4.2 release note is not mentioned about the issue below? It is fixed ?

Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. mmap() failed: Out of memory size: 1207959552 offset: 0 in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 217
       at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
       at io.realm.internal.SharedRealm.<init>(SharedRealm.java:190)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:237)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:206)
       at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:127)
       at io.realm.Realm.getDefaultInstance(Realm.java:266)
       at com.xxx.xxx.data.source.local.RealmProcessor.init(RealmProcessor.java:29)
       at com.xxx.xxx.xxxApplication.onCreate(xxxApplication.java:41)
       at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1014)
       at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4804)
       at android.app.ActivityThread.access$1600(ActivityThread.java:154)
       at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1452)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:234)
       at android.app.ActivityThread.main(ActivityThread.java:5524)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)

realm / realm-java

io.realm.exceptions.RealmFileException when launching application #4524