Updating local mbtiles file causes crash

[nnhubbard]

I am using the mbtiles:// local file feature and I am running into an issue. In my app, I allow downloading of world-wide maps in mbtiles format. Updated maps are generated on my server, so users can choose to download an updated map.

The crash occurs when user has a specific map loaded, and then they download an updated map of that loaded location and the file on the file system gets replaced. This causes a crash because the database for the mbtiles file is not closed before updating the file.

Crash:

com.mapbox.mbgl.MaptilerFileSource (12): EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
terminating with uncaught exception of type mapbox::sqlite::Exception

Replicating Crash:

• Use the iosapp target after using make iproj.
• Add an mbtiles file to your project such as from openmaptiles.org.
• Add a style.json file to your project such as https://github.com/openmaptiles/maptiler-basic-gl-style
• Load the styles and write the mbtiles path into the style. Update the MBXViewController.m file of the Demo app with the following:

MBXViewController.m add:

- (NSURL *)writeMbtilesToCreatePath:(NSData *)jsonData {

    NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
    NSString *documentsDirectory = [paths objectAtIndex:0]; // Get documents directory

    NSString *styleURL = [documentsDirectory stringByAppendingPathComponent:@"mbtileStyle.json"];
    BOOL succeed = [jsonData writeToFile:styleURL atomically:YES];
    if (succeed){

        styleURL = [@"file:///" stringByAppendingString:styleURL];
        return [NSURL URLWithString:styleURL];

    }

    return nil;

}

- (NSData *)mbtilesStyleJSONWithFilenames:(NSString *)filename {

    // Style Path
    NSString *masterStyles = [[NSBundle mainBundle] pathForResource:@"style" ofType:@"json"];
    if (DEBUG) {
        NSLog(@"Mapbox Style: %@", masterStyles);
    }

    // Create dictionary of style JSON
    NSError *error = nil;
    NSDictionary *json = [NSJSONSerialization JSONObjectWithData:[NSData dataWithContentsOfFile:masterStyles] options:NSJSONReadingAllowFragments error:&error];
    if (error) {
        NSLog(@"Load Style JSON Error: %@", error);
    }

    // JSON Source
    NSMutableDictionary *mutableJson = [json mutableCopy];
    NSMutableDictionary *sources = [[NSMutableDictionary alloc] init];

    /// .mbtiles File paths
    NSString *databasePath = [[NSBundle mainBundle] pathForResource:filename ofType:@"mbtiles"];;

    // Create the mbtiles json
    NSMutableDictionary *openMapTilesSource = [[NSMutableDictionary alloc] init];
    openMapTilesSource[@"url"] = [NSString stringWithFormat:@"mbtiles://%@", databasePath];
    openMapTilesSource[@"type"] = @"vector";
    openMapTilesSource[@"maxzoom"] = @(14);

    // Update Sources
    sources[@"openmaptiles"] = openMapTilesSource;

    // Update Mutable Layers
    mutableJson[@"layers"] = json[@"layers"];

    // Update sources
    mutableJson[@"sources"] = sources;

    if (!mutableJson) {
        return nil;
    }
    NSError *error2;
    NSData *jsonData = [NSJSONSerialization dataWithJSONObject:mutableJson options:NSJSONWritingPrettyPrinted error:&error2];

    return jsonData;

}

MBXViewController.m in viewDidLoad around line 279 add:

NSData *jsonStyles = [self mbtilesStyleJSONWithFilenames:@"map"];
NSURL *styleURL = [self writeMbtilesToCreatePath:jsonStyles];
if (styleURL) {
        self.mapView.styleURL = styleURL;
 }

Run the test app

Browse the file system for the MapLibre GL.app and view the contents of the app to find your mbtiles file. Make a duplicate somewhere and then overwrite the original file within the MapLibre GL.app folder.

Moving the map in the test app will now cause a crash.

[roblabs]

I am able to reproduce the crash on iOS and I have a branch at roblabs/maplibre-gl-native:100-updating-local-mbtiles-file-causes-crash that has this sample code for reproducibility and a brute force, but working, fix.

Exception is unhandled due to the local MBTiles file being updated before it was closed properly. Use case will affect any app that uses MBTiles AND updates the same map that is being displayed.

Next Steps:

• [x] Create a first pass fix (although, it's not the right fix as it closes the DB every time a tile is accessed)
• [ ] Have MapTiler team review for suggested fixes on how to close the SQLite3 DB so that an MBTiles can be updated without restarting the app. This is not specific to MapTiler made MBTiles; it will affect any MBTiles (just looking for review or guidance).
• [ ] Possibly trap for SQLite3 error codes that indicate that the SQLite3 DB should be closed, that way we can continue gracefully.

---

Error Code Details

The crash is due to a thrown exception because an SQL error code came through that is not SQLITE_OK

https://github.com/maplibre/maplibre-gl-native/blob/ee94520bffeae2f6246284e1931d0cc1d615f708/platform/default/src/mbgl/storage/sqlite3.cpp#L335-L337

This throws because the error code that came through is: (6922) SQLITE_IOERR_VNODE

2021-06-28 15:07:55.391756-0700 MapLibre GL[4015:2042647] [DEBUG] {com.mapbox.mbgl.MaptilerFileSou}[Database](6922): statement aborts at 23: [SELECT tile_data FROM tiles where zoom_level = 2 AND tile_column = 1 AND tile_row = 2] disk I/O error

Error code 6922 details:

(6922) SQLITE_IOERR_VNODE The SQLITE_IOERR_VNODE error code is a code reserved for use by extensions. It is not used by the SQLite core.

Source: SQLite Result and Error Codes

• Style is MapTiler Basic
• MBTiles is from

  wget https://github.com/maptiler/tileserver-gl/releases/download/v1.3.0/zurich_switzerland.mbtiles

[dieterdreist]

I also get exceptions with mbtiles files. Errors are like this, but only occur (as far as I have observed) when debugging (run app via XCode), they do not happen when I open the app on the device:

[logging] BUG IN CLIENT OF libsqlite3.dylib: database integrity compromised by API violation: vnode unlinked while in use: <unresolvable path>
[logging] invalidated open fd: 9 (0x11)
libc++abi: terminating with uncaught exception of type mapbox::sqlite::Exception
terminating with uncaught exception of type mapbox::sqlite::Exception