Jarred-Sumner commented 4 years ago

MMKV: cross-platform mobile key/value store made by Tencent https://github.com/Tencent/MMKV JSI: JavaScript interface(?) -- its the base of React Native's new architecture that lets you skip the bridge.

Motivation

The bridge in React Native is really slow, and maintaining your own database is hard & not very fun (for me, at least).

Description

Instead of doing all that, you could just wrap MMKV and use the JSI to skip the bridge. It also makes it synchronous, which is actually good considering how fast it is.

I built this in my app.

Benchmarks of MMKV from their github repo, as compared to NSUserDefaults:

I haven't benchmarked the difference between AsyncStorage and MMKV, but my app felt noticeably faster after I switched and that's all I cared about. It probably is a bad idea if the file size exceeds 100MB, since its a memory-mapped database (so everything is stored in memory & synced automatically to disk)

New feature implementation

Here's example code ripped out from my app where I have this working (renamed some things).

In JavaScript, this code is called via:

global.YeetStorage.getItem(key, type)
global.YeetStorage.setItem(key, value, type)

global.YeetStorage.removeItem(key)


//
//  StorageModule.m
//  yeet
//
//  Created by Jarred WSumner on 1/29/20.
//  Copyright © 2020 Yeet. All rights reserved.
//

import "StorageModule.h"

import <ReactCommon/TurboModule.h>

import <Foundation/Foundation.h>

import <MMKV/MMKV.h>

@interface RCTBridge (ext)

(std::weak_ptr)reactInstance; @end

StorageModule::StorageModule(RCTCxxBridge *bridge) : bridge_(bridge) { std::shared_ptr _jsInvoker = std::make_shared(bridge.reactInstance); }

void StorageModule::install(RCTCxxBridge *bridge) { if (bridge.runtime == nullptr) { return; }

jsi::Runtime &runtime = (jsi::Runtime )bridge.runtime;

auto reaModuleName = "YeetStorage"; auto reaJsiModule = std::make_shared(std::move(bridge)); auto object = jsi::Object::createFromHostObject(runtime, reaJsiModule); runtime.global().setProperty(runtime, reaModuleName, std::move(object)); }

jsi::Value StorageModule::get(jsi::Runtime &runtime, const jsi::PropNameID &name) { auto methodName = name.utf8(runtime);

if (methodName == "removeItem") { MMKV mmkv = [MMKV defaultMMKV]; return jsi::Function::createFromHostFunction(runtime, name, 1, [mmkv]( jsi::Runtime &runtime, const jsi::Value &thisValue, const jsi::Value arguments, size_t count) -> jsi::Value {

  NSString *key = convertJSIStringToNSString(runtime, arguments[0].asString(runtime));

  if (key && key.length > 0) {
    [mmkv removeValueForKey:key];
    return jsi::Value(true);
  } else {
    return jsi::Value(false);
  }
});

} else if (methodName == "getItem") { MMKV mmkv = [MMKV defaultMMKV]; return jsi::Function::createFromHostFunction(runtime, name, 2, [mmkv]( jsi::Runtime &runtime, const jsi::Value &thisValue, const jsi::Value arguments, size_t count) -> jsi::Value {

  NSString *type = convertJSIStringToNSString(runtime, arguments[1].asString(runtime));
  NSString *key = convertJSIStringToNSString(runtime, arguments[0].asString(runtime));

  if (!key || ![key length]) {
    return jsi::Value::null();
  }

  if ([type isEqualToString:@"string"]) {
    NSString *value = [mmkv getStringForKey:key];

    if (value) {
      return convertNSStringToJSIString(runtime, value);
    } else {
      return jsi::Value::null();
    }
  } else if ([type isEqualToString:@"number"]) {
    double value = [mmkv getDoubleForKey:key];

    if (value) {
      return jsi::Value(value);
    } else {
      return jsi::Value::null();
    }
  } else if ([type isEqualToString:@"bool"]) {
    BOOL value = [mmkv getBoolForKey:key defaultValue:NO];

    return jsi::Value(value == YES ? 1 : 0);
  } else {
    return jsi::Value::null();
  }
});

} else if (methodName == "setItem") { MMKV mmkv = [MMKV defaultMMKV]; return jsi::Function::createFromHostFunction(runtime, name, 3, [mmkv]( jsi::Runtime &runtime, const jsi::Value &thisValue, const jsi::Value arguments, size_t count) -> jsi::Value { NSString type = convertJSIStringToNSString(runtime, arguments[2].asString(runtime)); NSString key = convertJSIStringToNSString(runtime, arguments[0].asString(runtime));

  if (!key || ![key length]) {
    return jsi::Value::null();
  }

  if ([type isEqualToString:@"string"]) {
    NSString *value = convertJSIStringToNSString(runtime, arguments[1].asString(runtime));

    if ([value length] > 0) {
      return jsi::Value([mmkv setString:value forKey:key]);
    } else {
      return jsi::Value(false);
    }
  } else if ([type isEqualToString:@"number"]) {
    double value = arguments[2].asNumber();

    return jsi::Value([mmkv setDouble:value forKey:key]);
  } else if ([type isEqualToString:@"bool"]) {
    BOOL value = arguments[2].asNumber();

    return jsi::Value([mmkv setBool:value forKey:key]);
  } else {
    return jsi::Value::null();
  }
});

}

return jsi::Value::undefined(); }


```objc++
//
//  StorageModule.h
//  yeet
//
//  Created by Jarred WSumner on 2/6/20.
//  Copyright © 2020 Yeet. All rights reserved.
//

#import <jsi/jsi.h>
#include <ReactCommon/BridgeJSCallInvoker.h>

using namespace facebook;

@class RCTCxxBridge;

class JSI_EXPORT StorageModule : public jsi::HostObject {
public:
    StorageModule(RCTCxxBridge* bridge);

    static void install(RCTCxxBridge *bridge);

    /*
     * `jsi::HostObject` specific overloads.
     */
    jsi::Value get(jsi::Runtime &runtime, const jsi::PropNameID &name) override;

    jsi::Value getOther(jsi::Runtime &runtime, const jsi::PropNameID &name);

private:
    RCTCxxBridge* bridge_;
    std::shared_ptr<facebook::react::JSCallInvoker> _jsInvoker;
};

You would override setBridge in the RCTBridgeModule and call StorageModule::install(self.bridge);

There's a better implementation here too, where you skip the Foundation primitives and use C++ directly. If you did that, you wouldn't pay the cost of converting from std::string NSString.

sebqq commented 4 years ago

Hello @Jarred-Sumner, I think this is good idea overal, but sometimes, when you need to push a big chunk of data it could be actually usefull to use async option, so your JS thread does not get stucked. Did you benchmark even bigger data on low end android devices?

I think that user should still have possibility to use async version of ‘setItem’ because we dont always expect data to be available immediately after writing them to storage.

krizzu commented 4 years ago

Hey @Jarred-Sumner ,

I've marked this issue as v2, as this looks like a good candidate for a new storage backend for v2.

I'd like to give it a try, benchmarks are promising, but I have no experience with MMKV. My main question for implementation would be:

Are types for values only limited to strings, numbers and booleans? What about arrays and objects?
What are known limitations (asking mainly about the max size it can store, but would love it know if there's anything else to know)
Haven't followed the JSI development, so do you know if the API is going to change in the future?

Jarred-Sumner commented 4 years ago

The storage backend vs JSI are kind of separate things. MMKV is a database and the JSI is more like the React Native Bridge -- one is for persistence and the other is for sending data between JavaScript and native code. MMKV as a storage backend sounds like a good idea though!

Are types for values only limited to strings, numbers and booleans? What about arrays and objects?

On iOS, anything that implements NSCodable (which includes NSDictionary and NSArray, so yes) which is kind of the Objective C-specific way of encoding/decoding data – https://github.com/Tencent/MMKV/wiki/iOS_tutorial#supported-types

On Android, it sounds like yes (Parcelable) but I'm not as familiar with Android/Java: https://github.com/Tencent/MMKV/wiki/android_tutorial#supported-types

What are known limitations for MMKV (asking mainly about the max size it can store, but would love it know if there's anything else to know)

From MMKV's FAQ:

MMKV works perfectly well on most case, the size and length of keys and values is unlimited. However, since MMKV caches everything in memory, if the total size is too big (like 100M+), App may receive memory warning. And write speed might slow down when a full write back is needed. We do have a plan to supporting big files, hopefully will come out in next major release.

Haven't followed the JSI development, so do you know if the API is going to change in the future?

I know that they renamed one of the header files & classes between 0.61 and 0.62 (JSCallInvoker -> CallInvoker). That was the only thing I had to change for 0.62 though. I don't work on React Native itself so I don't really know how much to expect it to change.

chr4ss1 commented 4 years ago

Interesting! I am just in the middle of benchmarking and testing MMKV. It would be interesting to see how JSI will perform so I am also going to go through and test it on lower end Android devices and on iOS.

Having MMKV as the default storage could make a lot of sense, given that it is used by WeChat (200m+ users). It also seems to be having CRC32 checks, and the corruption rate of storage is 20 users out of 200m.

@Jarred-Sumner few questions

1) what did you mean by "If you did that, you wouldn't pay the cost of converting from std::string NSString" -> Where would you pass that std::string? MMKV seems to be built on top of foundation classes (NSString).

The reason why I am asking is because there is going to be quite a lot of conversions going on when you are accessing items from MMKV. (From NSString->std::string).

My current solution is just a simple cache on JS side for AsyncStorage. The read is going to be faster than JSI+MMKV because well, there is no need to deserialize anything, and no conversions needed as well.

My application is not that write heavy, but quite read heavy

import ParseService from './ParseService';
import LogService from './LogService';
import { AsyncStorage } from 'react-native';
import * as _ from 'lodash';

const userDataKey = 'userData';
const userDataGlobalPrefixKey = 'userData';

function jsonStringify (data) { return JSON.stringify(data); }
function jsonParse (data) {

    const jsonReplacerInternal = (key, value) => {
        const reISO = /^\d{4}-(0[1-9]|1[0-2])-([12]\d|0[1-9]|3[01])([T\s](([01]\d|2[0-3])\:[0-5]\d|24\:00)(\:[0-5]\d([\.,]\d+)?)?([zZ]|([\+-])([01]\d|2[0-3])\:?([0-5]\d)?)?)?$/;

        if (typeof value === 'string') {
            // then, use regex to see if it's an ISO-formatted string
            let a = reISO.exec(value);
            if (a) {
                // if so, Date() can parse it:
                return new Date(value);
            }
        }
        // important: you need to return any values you're not parsing, or they die...
        return value;
    };

    return JSON.parse(data, jsonReplacerInternal);
}

function serialize (data) { return jsonStringify(data); }
function deserialize (data) { return jsonParse(data); }

async function setItemInternal (key, data) {
    await AsyncStorage.setItem(key, serialize(data));
}

async function getItemInternal (key) {
    const item = await AsyncStorage.getItem(key);

    if (item) {
        return deserialize(item);
    }

    return null;
}

async function initializeGlobalCache () {
    if (!global[userDataGlobalPrefixKey]) {
        global[userDataGlobalPrefixKey] = {};
    }
}

export default {

    setItem: setItemInternal,
    getItem: getItemInternal,

    setUserData: async (key, data) => {

        if (!key) {
            throw new Error('Key is null');
        }

        const storageKey = `${userDataKey}_${key}`;

        initializeGlobalCache();

        global[userDataGlobalPrefixKey][storageKey] = data;

        await setItemInternal(storageKey, data);
    },

    getUserData: async (key, initializationValue) => {

        if (!key) {
            throw new Error('Key is null');
        }

        const storageKey = `${userDataKey}_${key}`;

        initializeGlobalCache();

        if (global[userDataGlobalPrefixKey][storageKey]) {
            return global[userDataGlobalPrefixKey][storageKey];
        }

        global[userDataGlobalPrefixKey][storageKey] = await getItemInternal(storageKey) || initializationValue;
        return global[userDataGlobalPrefixKey][storageKey];
    },

};

joshxyzhimself commented 4 years ago

Has anyone tried https://github.com/ammarahm-ed/react-native-mmkv-storage?

ricardobeat commented 3 years ago

@Krizzu why was this closed? Seems quite promising for performance (I'm looking at 10s+ of loading stuff from AsyncStorage at this very moment).

krizzu commented 3 years ago

Closed this due being related to non-existing not v2 version of Async Storage. We don't plan to replace the way how we store data in current version, hence I closed it.

ricardobeat commented 3 years ago

What's the best place to record feature requests / suggestions then?

safaiyeh commented 3 years ago

For anyone that stumbles on this thread a JSI mmkv module exists here: https://github.com/mrousavy/react-native-mmkv

kesha-antonov commented 2 years ago

Looks promising. Will try those libs mentioned

react-native-async-storage / async-storage

Migrate to the JSI & MMKV for better performance #291