Insert more then x documents blocks UI

[bartoszwrobel]

Hi, I have strange problem. This is my sample code:

import React, { Component } from 'react';
import {
    View,
    Text,
    TouchableNativeFeedback
} from 'react-native';
import Realm from 'realm';

Realm.clearTestState();

const ProductSchema = {
    name: 'Product',
    properties: {
        variantId: 'string',
        productId: 'string',
        productNamePL: 'string',
        productNameEN: { type: 'string', optional: true },
        variantNamePL: 'string',
        variantNameEN: { type: 'string', optional: true },
        type: { type: 'int', optional: true },
        unit: { type: 'int', optional: true },
        tax: { type: 'int', optional: true },
        pricePLN: { type: 'double', default: 0 },
        priceEUR: { type: 'double', default: 0 },
        priceUSD: { type: 'double', default: 0 },
        priceRUB: { type: 'double', default: 0 },
        priceCHF: { type: 'double', default: 0 },
        priceGBP: { type: 'double', default: 0 }
    }
};

const config = {
    schema: [ProductSchema],
};

let realm = new Realm(config);

export default class App extends Component {

    save() {
        console.log('start')
        try {
            realm.write(() => {
                for (const i = 0; i < 1175; i++) {
                    const product = {
                        variantId: 'variant' + i,
                        productId: 'product' + 1,
                        productNamePL: 'Variant name',
                        productNameEN: 'Variant name',
                        variantNamePL: 'Variant name',
                        variantNameEN: 'Variant name',
                        type: 1,
                        unit: 500,
                        tax: 8,
                        pricePLN: 10,
                        priceEUR: 8,
                        priceUSD: 5,
                        priceRUB: 9,
                        priceCHF: 3,
                        priceGBP: 12
                    };
                    realm.create('Product', product);
                }
            });
            console.log('done');
        } catch (e) {
            console.log(e);
        }

    }

    render() {
        return (
            <View style={{
                backgroundColor: 'white',
                flex: 1,
                alignItems: 'center',
                justifyContent: 'center',
            }}>
                <TouchableNativeFeedback
                    onPress={this.save}
                    background={TouchableNativeFeedback.SelectableBackground()}>
                    <View style={{
                        width: 150,
                        height: 100,
                        backgroundColor: 'red',
                        alignItems: 'center',
                        justifyContent: 'center'
                    }}>
                        <Text>Tap</Text>
                    </View>
                </TouchableNativeFeedback>
            </View>
        );
    }

};

The problem is, when I insert more then 1175 documents (e.g. call twice save() or change i < 1176) the react-native UI is blocked. Checked on different devices, always number of documents which block UI is the same. This is not a fault of documents count, but documents size. When change some strings length - can insert more documents, etc. UI blocked only when I exceed number of documents while using app. When i close app and run again, I can insert next 1175. Realm has some limitation per session or it's a bug? Realm 0.15.4 (the same problem on 0.14.3) React Native 0.38.0

[kristiandupont]

Does the UI block indefinitely, or does it just hang for a while? As javascript is single threaded, you might want to split your transactions up and do a number of them instead.

[bartoszwrobel]

UI block indefinitely, whole app is not responding, I can just kill it and restart. I tried to smaller transactions, but it's not helping. After crossing some size app just hang, no errors, no anything. You tried to run this code? How many times you can tap and call save()? I can call it twice, but when second call finished - app crashes.

[daominhsangvn]

I got the same problem. In my case, i tried to insert/update about 4000+ documents, the app get hang for a while then become normal. Any idea?

[yonedev]

I have the same problem.

[ruslanchek]

Same problem. Only 50 documents with 10 fields each (ints and booleans) in my DB. Any React Native component that interacts with a Realm causes 2-3 seconds freeze.

[daominhsangvn]

You guys need to use requestAnimationFrame to prevent the UI blocks.

# Example:
const data = [...];

const insertData = (index) => {
   const dt = data[index];

   if(!dt) return;

   // code insert data into realm here

   requestAnimationFrame(() => insertData(index + 1));
}

insertData(0);

[ruslanchek]

You guys need to use...

In my case i haven't any loop through objects and update/insert, just find by id then update one field. I don't think it's too heavy operation. It's literally most primitive operation for DB. Yes, i'm tried RAF, and no luck :-(

[nixoschu]

I stumbled upon the same problem today in this is how I solved it for me

Objects to save: 11883 Scenario: During my Intro slides of the App I need to download a catalogue of components for later usage. Writing all objects into the database in a single write operation caused my app to freeze , the operation was still performed tho.

For me the key was to use InteractionManager.runAfterInteractions and Split the write operations in smaller packages. I did not perform any calculation if the bulkSizecould be larger, since it worked fast enough I did not test any further.

My Dependencies

  "dependencies": {
    "i18n-js": "^3.1.0",
    "moment": "^2.23.0",
    "react": "16.6.1",
    "react-moment": "^0.8.4",
    "react-native": "0.57.7",
    "react-native-gesture-handler": "^1.0.10",
    "react-native-languages": "^3.0.2",
    "react-native-splash-screen": "^3.1.1",
    "react-native-swipe-list-view": "^1.5.0",
    "react-native-swiper": "^1.5.14",
    "react-navigation": "^3.0.5",
    "realm": "^2.22.0"
  },

  componentDidMount() {
    setTimeout(() => {
      this._swiperobj.scrollBy(1);
      setTimeout(() => {
        this._swiperobj.scrollBy(1);
      }, 2000);
    }, 2000);

    InteractionManager.runAfterInteractions(() => {
      ComponentApi.getAll()
        .then((componentData) => {
          const bulkSize = 100;
          const loops = Math.ceil(componentData.length / bulkSize);

          console.log(loops + ' needed to insert all components');

          for (var i = 1; i <= loops; i++) {
            console.log('#' + i + ' loop');
            const start = (i - 1) * bulkSize;
            const end = i * bulkSize;

            const bulkPartArray = componentData.slice(start, end);
            console.log('writing next components to database');
            this._writeComponentsToDatabase(bulkPartArray);
          }

          this.setState({ hasFinishedLoading: true });
        })
        .catch((error) => {
          console.log('ERROR LOADING COMPONENTS');
          console.log(error);
        });
    });
  }

  _writeComponentsToDatabase(components, callback) {
    // this.props.screenProps.db is my realm instance
    this.props.screenProps.db.write(() => {
      for (let i = 0; i < components.length; i++) {
        this.props.screenProps.db.create(Component.schema.name, components[i]);
      }
    });
  }

[jrounsav]

Running into this on 3.3.x while trying to write 10k contacts to the DB. Using InteractionManager and batching the writes don't seem to help at all either, as the write time increases by ~100ms every time. Freezing by the end. Batching in increments between 100 & 300.

I tried consolidating everything into a single write as well and the whole app froze. Checking Realm Studio shows that the data makes it in, and if I run realm.close() right after the write then the UI returns to normal. Restarting realm isn't an option though since it invalidates any existing collections.

[pedrocarlos-ti]

Same problem 😥😥

[Udbhav12]

Did anyone manage to solve this problem ?

I am trying to insert 25k records, where each record has 30 properties

It is taking around 30 seconds on Samsung Galaxy A10 device and also UI freezes temporarily when the write operation is being run

Is 30 seconds normal for 25k records ?

[pedrocarlos-ti]

Did anyone manage to solve this problem ?

I am trying to insert 25k records, where each record has 30 properties

It is taking around 30 seconds on Samsung Galaxy A10 device and also UI freezes temporarily when the write operation is being run

Is 30 seconds normal for 25k records ?

@Udbhav12 , can you share the code with us? It's not normal. In my app I have to save 40k with 15 - 20 properties in RealmDB and delay at most 3-4 seconds to save

[Udbhav12]

@pedrocarlos-ti - I am opening Realm synchronously via const realm = new Realm({schema: [PersonSchema]}); instead of Realm.open

Could this be a culprit ?

[pedrocarlos-ti]

@pedrocarlos-ti - I am opening Realm synchronously via const realm = new Realm({schema: [PersonSchema]}); instead of Realm.open

Could this be a culprit ?

Nope, it's the same way that I open my realm here. Can you share your block code where you call the realm method? I have a hunch of what might be.

[Udbhav12]

Nope, it's the same way that I open my realm here. Can you share your block code where you call the realm method? I have a hunch of what might be.

Below is the code which I use to insert records

const schemaVersion = 110;

const schema = [TABLE1 ,....];
var encryptionKey = new Int8Array(64);

let realm = new Realm({
    schemaVersion,
    schema,
    encryptionKey,
    shouldCompactOnLaunch: (totalBytes,usedBytes)=>{
        return true
    }
});

//array consists of 25k records where each record has 30 properties
array.forEach(data => realm.create(tableName, data, true));

[pbergner]

Any further development on this issue?

[takameyer]

@pbergner It was discussed today and we agree there should be a established way of performing large change operations in a way that does not block the UI. We will investigate some possible solutions.

[FSPinho]

For those who are still dealing with this issue, I got a small trick to work around this:

const runHeavyQuery = async () => {
    for (let i = 0; i < batchCount; i++) {
        realm.write(() => {
            // Your realm operations ....
        })

        await new Promise(resolve => setTimeout(resolve)); // <<< The trick
    }
}

This fake promise should give some breath to the JS execution queue. For your case, you'll need to find a balance between the write batch size, and the timeout delay (in my case ZERO was enough).

Of course, this approach will make your execution time increase, but at least the user will still be able to use the UI. In my case (200k+ items), for retrieve operations, I went from 11 to 13 secs, and for write operations, I went from 19 to 22 seconds.

[kneth]

await new Promise(resolve => setTimeout(resolve)); or even setTimeout(() => {}, 0); will kick the event loop and the UI will get a chance to update.

[cristiano-linvix]

@kneth any solution on new realm v12?

the UI is freezing because of data insert I'm inserting every 1000 records (total 100k+), but it's still blocking the UI.

Did I think of any multi thread alternatives?