search

realm / realm-java

Realm is a mobile database: a replacement for SQLite & ORMs
http://realm.io
Apache License 2.0
11.46k stars 1.75k forks source link

mmap() failed: io.realm.internal.SharedRealm.nativeGetSharedRealm #3629

Closed xlrei closed 4 years ago

xlrei commented 8 years ago

Realm Java 2.0.2

Fatal Exception: io.realm.exceptions.RealmError
Unrecoverable error. mmap() failed: Operation not permitted size: 589824 offset: 0 in /Users/cm/Realm/realm-java/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 81

My config is simple

@RealmModule (library = true, classes = { XLog.class })
private static class AnalyticsModule {}

final RealmConfiguration.Builder builder = new RealmConfiguration.Builder().name(FILE_NAME_FOR_XXX_ANALYTICS_REALM).schemaVersion(SCHEMA_VERSION_XXX_ANALYTICS).modules(new AnalyticsModule()).deleteRealmIfMigrationNeeded();

sRealmConfiguration = builder.build();

Anyone knows why？

beeender commented 8 years ago

Did you try to open a Realm which is on the external storage? It might be the same issue with #3140 Can you please share your RealmConfirugration?

xlrei commented 8 years ago

@beeender my SchemeVersion is 0 and never changed. I started use Realm from 0.8.x and I didn't edit relative code recently. Just Update Realm to 2.0.2, then it comes.

beeender commented 8 years ago

Yes, there is a bug #3140 introduced in recent versions that Realm file on external storage has problems.

Are you having Realm file on the external storage?

xlrei commented 8 years ago

@beeender I use default file config. Does not configure external storage.

beeender commented 8 years ago

Does it happen to all devices? Is it reproducible in your side? Can you share your apk with us? to help@realm.io

xlrei commented 8 years ago

@beeender I can't reproduce it. It is collect from Fabric and now there are two users effected by this exception in new version.

2016-10-14 1 42 59

xlrei commented 8 years ago

@beeender already send our app link to help@realm.io. Waiting for solution. Thx~

beeender commented 8 years ago

Comment from @finnschiermer :

Judging by the size of the mmap attempted, this is about mmapping the realm file (not the lock file or anything else) and it is reported as a permission problem (not a out of memory problem) so I'm guessing the file he is trying to open is (for some reason) now in a place where he does not have write access which does not sound like a Core problem

@xlrei Are you still meeting this issue after more users updated to the new version?

xlrei commented 8 years ago

@beeender This issue has few reports. So it is not a common problem. Maybe just HUAWEI' permission problem.

beeender commented 8 years ago

Thanks. I have tested on my Huawei honor 7, it works fine. I will try it if I can find a ALE-TL00 phone.

Anthonyeef commented 7 years ago

Found same issue on Huawei device. Exactly the same as description, and only found report from my user using Huawei Devices, like ATH AL00.

cmelchior commented 7 years ago

@xlrei @Anthonyeef Are you still seeing this problem?

mariusboepple commented 7 years ago

We're seeing this problem in our remote service which uses encrypted Realm.

#0. Crashed: main: 0 0 0x0000000000000000
       at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
       at io.realm.internal.SharedRealm.<init>(SharedRealm.java:192)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:240)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:208)
       at io.realm.RealmCache.doCreateRealmOrGetFromCache(RealmCache.java:297)
       at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:283)
       at io.realm.Realm.getInstance(Realm.java:292)
       at com.example.remotelib.data.DataBase.fetchSerials(DataBase.java:144)
       at com.example.remotelib.device.connection.BTConnection.startConnecting(BTConnection.java:602)
       at com.example.remotelib.device.connection.BTConnection.handleMessage(BTConnection.java:569)
       at com.example.remotelib.util.MainThreadHandler.handleMessage(MainThreadHandler.java:29)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:158)
       at android.app.ActivityThread.main(ActivityThread.java:7224)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1230)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1120)

--

Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. mmap() failed: Out of memory size: 603979776 in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 217
       at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
       at io.realm.internal.SharedRealm.<init>(SharedRealm.java:192)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:240)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:208)
       at io.realm.RealmCache.doCreateRealmOrGetFromCache(RealmCache.java:297)
       at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:283)
       at io.realm.Realm.getInstance(Realm.java:292)
       at com.example.remotelib.data.DataBase.fetchSerials(DataBase.java:144)
       at com.example.remotelib.device.connection.BTConnection.startConnecting(BTConnection.java:602)
       at com.example.remotelib.device.connection.BTConnection.handleMessage(BTConnection.java:569)
       at com.example.remotelib.util.MainThreadHandler.handleMessage(MainThreadHandler.java:29)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:158)
       at android.app.ActivityThread.main(ActivityThread.java:7224)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1230)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1120)

bildschirmfoto 2017-06-19 um 08 27 17

Zhuinden commented 7 years ago

I don't think multiprocess is officially supported with encryption, @beeender ?

beeender commented 7 years ago

@Zhuinden I think @MariusBoepple uses the encrypted Realm in the remote service process only. Otherwise it will crash early.

@MariusBoepple How big your Realm data could be? It seems Realm tries to mmap 603979776 bytes of virtual memory failed. Could your data be that big? 603 MB?

mariusboepple commented 7 years ago

@beeender You're right, we're using Realm only in the remote process.

And the data could not be that big - it should not be more than a few MB...

We faced a similar issue with the iOS version: https://github.com/realm/realm-cocoa/issues/4831

Zhuinden commented 7 years ago

I'd double-check if you actually close all local Realm instances that you open with realm.getDefaultInstance().

Generally, opening and not closing Realm instance on background thread pools such as Schedulers.io() can cause drastic increase in Realm files, because the thread-local versions they had the Realm opened on will not be cleared (because it is held open).

beeender commented 7 years ago

@MariusBoepple as @Zhuinden said, i think it has the same root cause with the same issue for your cocoa app -- on some non-looper thread, the Realm instance is not closed properly.

I am not saying that you don't have to close the Realm instance on a looper thread :) Generally if you don't need the Realm/RealmNotifications anymore, then you should close the Realm instance. But for looper thread, since we will deliver notifications and refresh the Realm instance on that thread, it won't hold a version of data forever. For non-looper thread, we cannot deliver notifications to it, so it will hold the version of data always, until you begin a transaction or call the refresh() manually.

mariusboepple commented 7 years ago

There are as many realm.close() as Realm.getInstance() so I assume that the instances are always closed properly. Mostly the instances are just opened for a few lines of code, like:

Realm realm = Realm.getInstance(getRealmConfig());
RealmResults<Car> cars = realm.where(Car.class).findAll();

ArrayList<String> numbers = new ArrayList<>();

for (Car car : cars) {
    String number = car.number;
    numbers.add(number);
}

realm.close();
return numbers;
Zhuinden commented 7 years ago

Can you guarantee that no exceptions occur? I always use try-finally for background threads.

beeender commented 7 years ago

If some Realm instances are not closed before they get GCed, you may see logs below when they get GCed:

            RealmLog.warn("Remember to call close() on all Realm instances. " +
                    "Realm %s is being finalized without being closed, " +
                    "this can lead to running out of native memory.", configuration.getPath()

Have you ever see this during developing & debugging?

mariusboepple commented 7 years ago

@beeender nope, didn't see anything like this lately...

beeender commented 7 years ago

@MariusBoepple you can call Realm.compact() to clean up the realm size before using Realm. I think that could solve the problem you have right now. Also, we will add an API compactOnLaunch() #3739 to make this easier, but it is not there right now.

But it is still quite important to know why that db file grows to that big. If you observed the abnormal size of Realm db, please let us know how to reproduce it. thanks a lot!

mariusboepple commented 7 years ago

We're using compactRealm() already every time the app is launched.

MikeKasperlik commented 7 years ago

As we still see this in our app we searched for a cause. We found that this crash peeks (extrem) when we do a new release. This has to do something with the customer installs the update over a recent version of the app. Maybe this helps.

beeender commented 7 years ago

@MikeKasperlik What is the backtrace for your crash? The issue you met sounds more like #2459

MikeKasperlik commented 7 years ago

This is the stacktrace:

Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. mmap() failed: Out of memory size: 872415232 in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 252
       at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java)
       at io.realm.internal.SharedRealm.<init>(SharedRealm.java:186)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:239)
       at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:202)
       at io.realm.RealmCache.doCreateRealmOrGetFromCache(RealmCache.java:298)
       at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:284)
       at io.realm.Realm.getInstance(Realm.java:301)
       at car.app.appkit.data.DataBase.fetchDongleSerials(DataBase.java:147)
       at car.app.appkit.device.connection.DeviceBTConnection.startConnecting(DeviceBTConnection.java:611)
       at car.app.appkit.device.connection.DeviceBTConnection.handleMessage(DeviceBTConnection.java:578)
       at car.app.appkit.util.MainThreadHandler.handleMessage(MainThreadHandler.java:29)
       at android.os.Handler.dispatchMessage(Handler.java:102)
       at android.os.Looper.loop(Looper.java:148)
       at android.app.ActivityThread.main(ActivityThread.java:7331)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1230)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1120)
beeender commented 7 years ago

@MikeKasperlik Your issue seems to be different. It should be caused by some Realm instances are not closed on the background thread. See https://realm.io/docs/java/latest/#large-realm-file-size

MikeKasperlik commented 7 years ago

I fear this is not the case. We do always close all realm instances immediately after finishing our db operations (checked this multiple times). We also monitor the open instances count with Realm.getGlobalInstanceCount() - This is always zero. We do use compactRealmOnLaunch. Maybe this is related?

galk-aleksandr commented 7 years ago

I think, we found same problem with Lenovo VIBE X3 Lite device. Stacktrace: Fatal Exception: io.realm.exceptions.RealmError: Unrecoverable error. Permission denied in /home/cc/repo/realm/release/realm/realm-library/src/main/cpp/io_realm_internal_SharedRealm.cpp line 252 at io.realm.internal.SharedRealm.nativeGetSharedRealm(SharedRealm.java) at io.realm.internal.SharedRealm.<init>(SharedRealm.java:186) at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:239) at io.realm.internal.SharedRealm.getInstance(SharedRealm.java:202) at io.realm.RealmCache.doCreateRealmOrGetFromCache(RealmCache.java:298) at io.realm.RealmCache.createRealmOrGetFromCache(RealmCache.java:284) at io.realm.Realm.getInstance(Realm.java:301) at com.yodhaapp.dailyhoroscope.realm.RealmTemplate.findInRealm(RealmTemplate.java:33) at com.yodhaapp.dailyhoroscope.localdevice.RealmLocalDeviceRepository.get(RealmLocalDeviceRepository.java:57) at com.yodhaapp.dailyhoroscope.localdevice.RealmLocalDeviceRepository.wasSyncedAtLeastOnce(RealmLocalDeviceRepository.java:78) at com.yodhaapp.dailyhoroscope.splash.SplashPresenter.attachView(SplashPresenter.java:67) at com.yodhaapp.dailyhoroscope.splash.SplashPresenter.attachView(SplashPresenter.java:27) at com.hannesdorfmann.mosby.mvp.delegate.MvpInternalDelegate.attachView(MvpInternalDelegate.java:63) at com.hannesdorfmann.mosby.mvp.delegate.ActivityMvpDelegateImpl.onCreate(ActivityMvpDelegateImpl.java:67) at com.hannesdorfmann.mosby.mvp.MvpActivity.onCreate(MvpActivity.java:42) at com.yodhaapp.dailyhoroscope.splash.SplashActivity.onCreate(SplashActivity.java:53) at android.app.Activity.performCreate(Activity.java:6220) at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1112) at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2513) at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2646) at android.app.ActivityThread.access$800(ActivityThread.java:182) at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1488) at android.os.Handler.dispatchMessage(Handler.java:111) at android.os.Looper.loop(Looper.java:194) at android.app.ActivityThread.main(ActivityThread.java:5763) at java.lang.reflect.Method.invoke(Method.java) at java.lang.reflect.Method.invoke(Method.java:372) at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:960) at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:755) 2017-09-08 8 41 10

Realm version: io.realm:realm-gradle-plugin:3.5.0

cmelchior commented 4 years ago

Closing due to being outdated.