io.realm.exceptions.RealmFileException when launching application

[jollyjoker992]

I have researched many and many times about this issue but I have not found any solution. I am implementing an Android application with Realm Android 3.0 and using Fabric to find out crash report. Below is Realm initialize code in Application class:

 Realm.init(this);
        RealmConfiguration config = new RealmConfiguration.Builder().name(REALM_SCHEMA_NAME)
                .schemaVersion(REALM_SCHEMA_VERSION)
                .migration(new Migration())
                .build();
        Realm.setDefaultConfiguration(config);
        Realm realm = Realm.getDefaultInstance(); // Automatically run migration if needed
        realm.close();

I get many crash report from Fabric related with io.realm.exceptions.RealmFileException such as:

• Unable to open a realm at path '/data/data/com.xxx/files/xxx.realm': Incompatible histories.
• Unable to open a realm at path '/data/data/com.xxx/files/xxx.realm': bad realm file header (#3). (bad realm file header (#3))
• Unrecoverable error. mmap() failed: Out of memory size xxx
• etc ...

So, can you tell me the reason of this issue? Thank you.

[Zhuinden]

@jollyjoker992 I mean that says your Realm file grew to 1207959552 bytes which means it was 1.125 GB large.

Is that normal for your amount of data?

[huunam118]

@Zhuinden , @cmelchior Thanks much for your contributing ! Currently, we try to reproduce this issue by trying to open too much transactions. But seem the realm files did not grow up even if it is crashed by 100 queued transactions. As we said before, we only used Realm on main thread. Any other idea about how the realm files size too large? Or have you got an example about that?

[Zhuinden]

@huunam118 you mean you do Realm.getDefaultInstance() on UI thread, and you only do realm.executeTransaction()?

[huunam118]

@Zhuinden Actually, we only call Realm.getDefaultInstance() on UI thread to get its instance. About transaction we used both executeTransactionAsync and executeTransaction, and there are some simple queries.

[jollyjoker992]

@Zhuinden @cmelchior I have a question relate this issue, pls help me.

Can I copy all data from old Realm file (corrupted with a large file size) to a new Realm file (create new file) without query and insert ? Of course, I want to keep both of them and switch to the expected file when working with them and just copy data (I don't want to clone a file with a large size).

Thank you

[beeender]

Can I copy all data from old Realm file (corrupted with a large file size) to a new Realm file (create new file) without query and insert ? Of course, I want to keep both of them and switch to the expected file when working with them and just copy data (I don't want to clone a file with a large size).

If it still can be opened and you just want to reduce the Realm file size, use Realm.compact() or RealmConfiguration.Builder.compactOnLaunch() will solve it.

But it is still important why the file grows too big, normally it is just because of Realm instances are not closed on the background thread.

[jollyjoker992]

@beeender : I think because I opened too many transaction at the same time so the Realm file grow up with the large size.

Realm.compact() or RealmConfiguration.Builder.compactOnLaunch()

cannot help me :(

Reference: https://github.com/realm/realm-java/issues/4524#issuecomment-348699329

[beeender]

I think because I opened too many transaction at the same time so the Realm file grow up with the large size.

So it is because of you really write that much of data to the Realm? One thing you can do is query for the data you don't need and delete them by RealmResults.deleteAllFromRealm() . The disk space can be reused later or if you want to reclaim the space immediately, you can call Realm.compactRealm() afterwards.

[jollyjoker992]

@beeender : No, I did not write too much data because the data is very simple. Reference to here, I think it's the root cause make my Realm file grow up with a large size. But I'm currently focusing on the issue that :

How can I copy all realm data from the old file (with the large size but has not corrupted) into a new file without query and insert.

[beeender]

No, I did not write too much data because the data is very simple. Reference to here, I think it's the root cause make my Realm file grow up with a large size.

If that is the case, Realm.compact() should just solve your issue. Have you tried that? it would be strange if it doesn't help.

Can the large Realm file still can be opened?

[jollyjoker992]

@beeender : I tried it but It cannot help me until the Realm file being corrupted.

[beeender]

If "Realm file being corrupted." means that it fails to open the file by mmap() failed xxx, then the file itself is not corrupted. It cannot be opened just because of there is not enough virtual memory space to mmap the Realm file. In this case, since you cannot open the file on that device (you probably can still open it on a 64bit device, but it is not practical for you I think.), you cannot query the data and rewrite it to another file.

[jollyjoker992]

@beeender : Thanks for your supporting. I tried to use the compact() method when launching the application but It seems cannot help me anything until the file being corrupted. So, when the compact() method should be called to compact Realm file? When launching app or when close app? And If the file grows up because of here, the compact() method can help me?

[beeender]

compact() should reclaim disk space which is not being used including the case you mentioned. See https://realm.io/docs/java/latest#faq-large-realm-file-size

compact operation may take some time to finish depends on how much valid data stared. Consider to use those APIs

https://realm.io/docs/java/4.3.1/api/io/realm/RealmConfiguration.Builder.html#compactOnLaunch--

https://realm.io/docs/java/4.3.1/api/io/realm/Realm.html#getInstanceAsync-io.realm.RealmConfiguration-io.realm.Realm.Callback-

[jollyjoker992]

@beeender : Should I compact when closing app? Ex: compact() then close Realm instance (I just use only 1 instance of Realm in my app)

[beeender]

Realm.getInstanceAsync() + Realm.Builder.compactOnLaunch() will be the my choice if I want to do that. But it depends on your use case if you can manage the app's life cycle well.

[jollyjoker992]

@beeender Thanks for your support. I will try it and feedback about the result for you ASAP. Many thanks from my heart :bowing_man:

[jollyjoker992]

@beeender Sorry because mention you but I need help from you. To detect and recover data after Realm file is corrupted, I need to make a Realm file that is corrupted for testing. So pls help me to know how to corrupt a Realm file manually? Thank you.

[bmunkholm]

@jollyjoker992 You can just provide any file that's not a realm file. An empty one for instance.

[jollyjoker992]

@bmunkholm I need a Realm crash when call Realm.init(Context). So, if I follow your solution, It's possible for testing?

[jollyjoker992]

@Zhuinden @kneth @cmelchior @beeender Sorry I need a help. If the Realm file is corrupted, the method Realm.init(appContext) will throw an Exception or only Realm.getInstance() will throw an Exception?

[cmelchior]

Only Realm.getInstance() . Realm.init() just load and setup the Realm library, so it isn't connected to opening any files.

[jollyjoker992]

@cmelchior Thanks. Sorry because I mentioned you many times. I am currently try to find the root cause of corrupted Realm file in my app. Thank you again.