realm / realm-js

Realm is a mobile database: an alternative to SQLite & key-value stores
https://realm.io
Apache License 2.0
5.78k stars 574 forks source link

Slow performance when inserting a tree of JSON data and backlinking #767

Open martigasco opened 7 years ago

martigasco commented 7 years ago

As suggested by a contributor in another issue #343 (about backlinking), I'm opening this due to a really slow performance when trying to store a reference to the parent in every children of a 3-level deep JSON tree data, apart from embedding each level of children to the parent.

For the sake of the issue, I prepared an example set of 3 levels of data (Continent,Country,City), with my corresponding Realm.js file where I set up the schemas, and the main React component where I perform the import. Please note that due to some refactoring to be able to post this here in a simple manner, some import statements might be misleading, but the important part is the fact that, once this example is up and running, it can take more than 10 minutes to insert, having each Level of the tree 6(Continent), 34(Country) and 466(City) items.

Realm.js - where I set up schemas

import Realm from 'realm';

class Continent {}
Continent.schema = {
  name: 'Continent',
  primaryKey: 'id',
  properties: {
    id: 'int',
    name:  'string',
    countries: {type: 'list', objectType: 'Country'}
  }
};

class Country {}
Country.schema = {
  name: 'Country',
  primaryKey: 'id',
  properties: {
    id: 'int',
    name:  'string',
    continent: 'Continent',
    cities: {type: 'list', objectType: 'City'}
  }
};

class City {}
City.schema = {
  name: 'City',
  primaryKey: 'id',
  properties: {
    id: 'int',
    name:  'string',
    country: 'Country'
  }
};

export default new Realm({
    schema: [Continent,Country,City], 
    schemaVersion: 3,
    migration: function(oldRealm, newRealm) {
        newRealm.deleteAll();
    }
});

MainComponent.js - where I perform the insert

import React, { Component } from 'react';
import { View, Text } from 'react-native';
import realm from './Realm';
import data from 'data.json';

class App extends Component {

    render() {

        realm.write(() => {
            for (var continent of data.continents) {
                var newContinent = realm.create('Continent', {name: continent.name, id: continent.id, countries: continent.countries}, true);
                console.log(newContinent.countries.length);
                for (var i = 0; i < newContinent.countries.length; i++) {
                    console.log('Country: '+newContinent.countries[i].name);
                    newContinent.countries[i].continent = newContinent;
                    for (var j = 0; j < newContinent.countries[i].cities.length; j++) {
                        console.log('City: '+newContinent.countries[i].cities[j].name);
                        newContinent.countries[i].cities[j].country = newContinent.countries[i];
                    }
                }
            }
        });

        return(
            <View style={{ paddingTop:35 }}>
                <Text>Count of Continents in Realm: {realm.objects('Continent').length}</Text>
                <Text>Count of Countries in Realm: {realm.objects('Country').length}</Text>
                <Text>Count of Cities in Realm: {realm.objects('City').length}</Text>
            </View>
        );
    }

}

export default App;

I know the nested for loops are not pretty, but for the sake of testing they prove my issue.

JSON data for the example:

data.json.zip

DISCLAIMER: I might be doing something wrong, so forgive me if I took a bad approach.

kristiandupont commented 7 years ago

Thank you for reporting this. I will look into it when I have some time.