crash when using Realm with tvOS

[mightyleader]

Hey, I'm looking at building a tvOS app, using my existing app logic. I use Realm for persistence in the iOS app. The process of removing unsupported frameworks etc.. is complete, but I get an exception thrown when i run it and it seems to point to Realm: 2015-09-23 10:47:24.549 noths[202:8789] * Terminating app due to uncaught exception 'RLMException', reason: 'open() failed: Operation not permitted' * First throw call stack: (0x18a508f50 0x19b2dbf80 0x100310e64 0x1003030e0 0x1003038ec 0x100303570 0x100303528 0x100235374 0x10023251c 0x1002323c8 0x1002879f0 0x1002877f0 0x100170f54 0x1002876cc 0x1000b3d68 0x1000ee898 0x1000ee4bc 0x18f8dc1a4 0x18f8dbd28 0x18f8e2b9c 0x18f8e003c 0x1886c9704 0x18f955450 0x1000b3bd4 0x18f950e84 0x18fb7e5dc 0x18fb82920 0x18fb7fa60 0x1934eb7ec 0x1934ebb6c 0x18a4c0598 0x18a4c002c 0x18a4bdd2c 0x18a3ecdd0 0x18f949d28 0x18f944c58 0x10021a998 0x19bb1e974) libc++abi.dylib: terminating with uncaught exception of type NSException

[mightyleader]

OK, looks like the open() call in C++ is going to butt up against the limitation on local storage in tvOS. I'm testing using Realm in memory instead of persisted to disk to see if that will work. Ideally I guess using iCloud/Ubiquity storage would probably get round the issue too.

[mrackwitz]

Yes, it's the limitation on local storage, which hits you there. In-memory Realms work fine as you figured out as well. But if you want to cache data locally on disk, you can also use the temporary directory like illustrated below and put your bet on the relatively conservative cache wipe frequency of the OS.

let paths = NSSearchPathForDirectoriesInDomains(.CachesDirectory, .UserDomainMask, true)
let cachesDirectory = paths.first! as NSString

let configuration = Realm.Configuration(path: cachesDirectory.stringByAppendingPathComponent("data.realm"))
Realm.Configuration.defaultConfiguration = configuration

[mightyleader]

Thanks, that's a good thought I'll give it a try. Do you know if storing Realms in the iCloud storage has been done successfully?

[mrackwitz]

Do you know if storing Realms in the iCloud storage has been done successfully?

No, I'm not aware of any real attempt in that direction. But it could work, given that only one device access the data.

[mightyleader]

Well I'll break some ground and report on it then ;)

On 23 Sep 2015, at 11:43, Marius Rackwitz notifications@github.com wrote:

Do you know if storing Realms in the iCloud storage has been done successfully?

No, I'm not aware of any real attempt in that direction. But it could work, given that only one device access the data.

― Reply to this email directly or view it on GitHub.

[jasarien]

I've been using this workaround on AppleTV in Provenance with great success up until just now. Apple released Beta 2 of tvOS and now it seems that Realm still throws an exception even when changing the default configuration to use the Caches directory...

If I use [RLMRealm realmWithConfig:error:]; then I get nil returned an an error object stating Error Domain=io.realm Code=2 "open() failed: No such file or directory" UserInfo={NSLocalizedDescription=open() failed: No such file or directory, Error Code=2}

The code I use:

    RLMRealmConfiguration *config = [[RLMRealmConfiguration alloc] init];
#if TARGET_OS_TV
    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSCachesDirectory, NSUserDomainMask, YES);
#else
    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
#endif
    [config setPath:[[paths firstObject] stringByAppendingPathComponent:@"default.realm"]];
    [RLMRealmConfiguration setDefaultConfiguration:config];
    self.realm = [RLMRealm defaultRealm];

The config I'm using for the defaultConfig:

RLMRealmConfiguration {
    path = /var/mobile/Containers/Data/Application/DA6DE009-6DB1-4E4F-8F76-B986E13FE97F/Library/Caches/default.realm;
    inMemoryIdentifier = (null);
    encryptionKey = (null);
    readOnly = 0;
    schemaVersion = 0;
    migrationBlock = (null);
    dynamic = 0;
    customSchema = (null);
}

The backtrace:

(lldb) bt
* thread #1: tid = 0x2f9c, 0x0000000192e5bf48 libobjc.A.dylib`objc_exception_throw, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
    frame #0: 0x0000000192e5bf48 libobjc.A.dylib`objc_exception_throw
    frame #1: 0x000000010055bb14 Provenance`RLMSetErrorOrThrow(error=<unavailable>, outError=<unavailable>) + 176 at RLMUtil.mm:276
    frame #2: 0x000000010054a4e0 Provenance`-[RLMRealm initWithPath:key:readOnly:inMemory:dynamic:error:](self=<unavailable>, _cmd=<unavailable>, path=<unavailable>, key=<unavailable>, readonly=<unavailable>, inMemory=<unavailable>, dynamic=<unavailable>, outError=<unavailable>) + 1984 at RLMRealm.mm:269
    frame #3: 0x000000010054aff4 Provenance`+[RLMRealm realmWithConfiguration:error:](self=<unavailable>, _cmd=<unavailable>, configuration=<unavailable>, error=0x0000000000000000) + 672 at RLMRealm.mm:400
    frame #4: 0x000000010054a870 Provenance`+[RLMRealm defaultRealm](self=<unavailable>, _cmd=<unavailable>) + 84 at RLMRealm.mm:302
  * frame #5: 0x000000010004e45c Provenance`-[PVGameLibraryViewController initWithCoder:](self=0x000000015e6938b0, _cmd="initWithCoder:", aDecoder=0x000000015f02e400) + 736 at PVGameLibraryViewController.m:87
    frame #6: 0x0000000187909d28 UIKit`-[UIClassSwapper initWithCoder:] + 248
    frame #7: 0x0000000187a2506c UIKit`UINibDecoderDecodeObjectForValue + 672
    frame #8: 0x0000000187a24db4 UIKit`-[UINibDecoder decodeObjectForKey:] + 336
    frame #9: 0x00000001879099cc UIKit`-[UIRuntimeConnection initWithCoder:] + 188
    frame #10: 0x000000018790a16c UIKit`-[UIRuntimeEventConnection initWithCoder:] + 68
    frame #11: 0x0000000187a2506c UIKit`UINibDecoderDecodeObjectForValue + 672
    frame #12: 0x0000000187a251e4 UIKit`UINibDecoderDecodeObjectForValue + 1048
    frame #13: 0x0000000187a24db4 UIKit`-[UINibDecoder decodeObjectForKey:] + 336
    frame #14: 0x0000000187908d00 UIKit`-[UINib instantiateWithOwner:options:] + 1220
    frame #15: 0x0000000187b5df08 UIKit`-[UIStoryboard instantiateViewControllerWithIdentifier:] + 196
    frame #16: 0x0000000187b5e068 UIKit`-[UIStoryboard instantiateInitialViewController] + 68
    frame #17: 0x00000001876ecfd8 UIKit`-[UIApplication _loadMainStoryboardFileNamed:bundle:] + 108
    frame #18: 0x00000001874b987c UIKit`-[UIApplication _loadMainInterfaceFile] + 264
    frame #19: 0x00000001876ebec8 UIKit`-[UIApplication _runWithMainScene:transitionContext:completion:] + 1316
    frame #20: 0x00000001876e9160 UIKit`-[UIApplication workspaceDidEndTransaction:] + 168
    frame #21: 0x000000018b06f7ec FrontBoardServices`-[FBSSerialQueue _performNext] + 184
    frame #22: 0x000000018b06fb6c FrontBoardServices`-[FBSSerialQueue _performNextFromRunLoopSource] + 56
    frame #23: 0x0000000182008528 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 24
    frame #24: 0x0000000182007fbc CoreFoundation`__CFRunLoopDoSources0 + 540
    frame #25: 0x0000000182005cbc CoreFoundation`__CFRunLoopRun + 724
    frame #26: 0x0000000181f34ce0 CoreFoundation`CFRunLoopRunSpecific + 384
    frame #27: 0x00000001874b2a80 UIKit`-[UIApplication _run] + 460
    frame #28: 0x00000001874ad9b0 UIKit`UIApplicationMain + 204
    frame #29: 0x0000000100093d7c Provenance`main(argc=1, argv=0x000000016fddfa68) + 124 at main.m:14
    frame #30: 0x000000019369e974 libdyld.dylib`start + 4

I've just verified in a separate test project that writing to the Caches directory is still possible, so I believe tvOS Beta 2 has introduced or changed something that is causing Realm to misbehave.

[mrackwitz]

@jasarien: Thanks for reporting back. We'll investigate that and will report, as soon as we know more.

[mightyleader]

Awesome, we're hit by the same thing now so we're a bit stuck :)

[jasarien]

Thanks, looking forward to it. (Reposted, as I originally posted on the wrong account >.< )

@mightyleader Glad I'm not alone.

[kishikawakatsumi]

The cause of the error is failing to create named pipe for inter-thread and inter-process notification, though I'm in the middle of the investigation.

Realm will create RLMNotifier instance when initializing itself, https://github.com/realm/realm-cocoa/blob/master/Realm/RLMRealm.mm#L450-L452

but instantiating RLMNotifier is failed due to mkfifo() (named pipe) function is not allowed on actual devices from beta2. https://github.com/realm/realm-cocoa/blob/cf1d33bf485516ec2992397017377551840650fd/Realm/RLMRealmUtil.mm#L192-L206

[jasarien]

What changed in b2?

In b1 I tested what could be written to on disk, the documents directory was prohibited. Caches was allowed and realm worked on b1. I didn't think to check the Library directory.

Where does realm try to write the named pipe? As far as I can see in my own testing Caches is still writable in b2.

[mightyleader]

Hey, Just to note we're using in memory storage until I get a chance to see if Ubiquity works and we get the same issue so it may not be linked to file locations

[jasarien]

Even with an in-memory realm I think the named pipe is still written to disk.

[mightyleader]

@jasarien ahh makes sense that it does that somewhere along the line yeah.

[kishikawakatsumi]

Realm creates FIFO special file for named pipe to the path same as data file. https://github.com/realm/realm-cocoa/blob/cf1d33bf485516ec2992397017377551840650fd/Realm/RLMRealmUtil.mm#L189

It is created even in-memory Realm for inter-process(and thread) notification. https://github.com/realm/realm-cocoa/blob/master/Realm/RLMRealm.mm#L449-L452

[jasarien]

So if it's the same path as the data file, which after setting the config should be the caches directory in my case, why does the write fail? I already verified that the caches directory was writable. Is this a limitation the OS is setting on that particular API?

Edit---

I went back and re-read your previous post where you stated mkfifo() is disallowed, not the writability. Apologies for misunderstanding.

[kishikawakatsumi]

I confirmed both NSTemporaryDirectory() and Library/Caches are still writable. It seems that mkfifo() function and mknod() function are not permitted any directory from beta2.

@jasarien Your above error https://github.com/realm/realm-cocoa/issues/2559#issuecomment-142710156 shows "open() failed: No such file or directory". I cannot reproduce the same error. Could you please make sure that error happens in beta2?

[jasarien]

@kishikawakatsumi That was the localzedDescription of the NSError object returned from [RLMRealm realmWithConfig:error:];

[jasarien]

I confirmed both NSTemporaryDirectory() and Library/Caches are still writable. It seems that mkfifo() function and mknod() function are not permitted any directory from beta2.

Do we know if there will be a fix for this? Or is Realm no longer usable at all on tvOS?

I am raising a bug with Apple about this, since I believe this to be a bug that will break many apps that use Realm. I encourage Realm to do the same so that it will be brought to their attention. We should state in the bug reports how many apps use Realm (estimated) and how difficult a work around would be to implement.

I have raised bug 22847480 with Apple. Please duplicate this bug and include as much detail about it as possible.

[mrackwitz]

Do we know if there will be a fix for this? Or is Realm no longer usable at all on tvOS?

No named pipes won't be from my understanding necessarily a blocker, even if that's a definitive decision that they are not available. I can't make up yet a scenario where inter-process notifications could be actually relevant on AppleTV, at least with the current possibilities on the platform. So we could use a pure runtime-based inter-thread notifier as alternative implementation, as we did before inter-process support. I updated the seg-tvosbranch accordingly.

[jasarien]

Thanks I will check it out. In the meantime, I would still strongly encourage Realm to create a duplicate of the bug I logged with Apple.

[jpsim]

Just a reminder that although we're actively pursuing the viability of officially supporting tvOS, this is still very much in the early experimental stages. We'll help out anyone we can, but our top priority continues to be supporting our existing, officially-supported releases.

[jasarien]

That's understood and I really appreciate the help I've already received.

With that said, I still think it would be in the best interests of Realm, its users and the rest of the Apple development community if this bug was chased with Apple. The more control they wrestle from us in terms of limiting API like this, the less creative we'll be as a community.

[jpsim]

100% agreed. Radar filed: http://www.openradar.me/22860545

[jasarien]

Awesome :D

[jasarien]

I tried to build the seg-tvos branch this evening and I've run into the error:

RLMRealm.mm:249:113: error: no member named 'get_path' in 'realm::util::File::PermissionDenied'
        error = RLMMakeError(RLMErrorFilePermissionDenied, File::PermissionDenied(newMessage.UTF8String, ex.get_path()));

This happens when building both the iOS and tvOS framework target in Xcode, and also when running the sh build.sh build command from the command line.

Have I missed something? The Realm core binaries appear to be downloaded (there is a directory named core-0.92.2 with binaries inside).

[jpsim]

@jasarien the seg-tvos branch is not ready to be built by the general public at the time being as it depends on some private builds of Realm's core C++ library.

[jasarien]

:( Is there anything else I can do to test out Realm on tvOS beta 2?

[jpsim]

@jasarien I've updated #2506 with tvOS builds of Realm's C++ core, along with the following note in the PR:

Note to Realm users: You may use this branch to begin preliminary tvOS development, but these builds are not suitable for production use. Expect this branch to be rebased at any time, and features could be added/removed at any point in time.

[jasarien]

@jpsim Thank you! I really appreciate all the help you and the other Realm peeps have given over the last few days.

[jpsim]

Closing as this specific issue is resolved in #2506. If you encounter tvOS-specific bugs in the future, please file an issue stating exactly what it is, keeping in mind that we don't officially support the platform at the moment.