realm / realm-js

Realm is a mobile database: an alternative to SQLite & key-value stores
https://realm.io
Apache License 2.0
5.72k stars 564 forks source link

> I completely forgot I had opened this issue. I did manage to fix this back in the day but forgot to update... Back then I seemed to be the only one in the whole world with this problem :P #6765

Closed stee1ix closed 1 month ago

stee1ix commented 2 months ago
          > I completely forgot I had opened this issue. I did manage to fix this back in the day but forgot to update... Back then I seemed to be the only one in the whole world with this problem :P

So, basically I ended up using an approach along the lines of what others have been suggesting in this thread, like in the previous comment.

My trick was to leverage React Native's Timers API, more specifically the "requestAnimationFrame" method. Additionally, I had to come up with a way to split my data into chunks to make "create" transactions a bit lighter, instead of trying to create tens of thousands of object at once.

Her's how the code looks like:

function nextFrame() {
  return new Promise<void>(function(resolve, _reject) {
    requestAnimationFrame(function() {
      resolve();
    });
  });
}

async function storeRecords(records: Array<Record>): Promise<void> {
  try {
    const realm = await openRecordRealm();
    const chunkSize = 1000;

    if (records.length > chunkSize) {
       const chunkArray: Record[][] = [];

      for (let i = 0; i < records.length; i += chunkSize) {
        chunkArray.push(records.slice(i, i + chunkSize));
      }

      realm.beginTransaction();
      try {
        for (let i = 0; i < chunkArray.length; i++) {
          chunkArray[i].forEach(dbRecord => {
            realm.create(RecordSchema.name, dbRecord, Realm.UpdateMode.All);
          });

          await nextFrame();
        }
        realm.commitTransaction();
      } catch (error) {
        realm.cancelTransaction();
        throw e;
      }
    } else {
      realm.write(() => {
        records.forEach(dbItem => {
          realm.create(RecordSchema.name, dbRecord, Realm.UpdateMode.All);
        });
      });
    }
  } catch (error) {
    console.error("Something went wrong while inserting records into DB", error);
  }
}

Here we have two functions: "nextFrame" and "storeRecords". The second is the one that handles the Realm object creation logic:

First I split my data set into chunks, in my case I had around 10 000 records to store and splitting them into chunks of 1 000 seemed to yield the best results. You have to play around with this number as it depends on how complex your data is, i.e., I tried with chunks of 5 000 and I still experience performance issues.

Now, the key is in the "nextFrame" function, whose only job is to resolve a promise function before the next UI repaint. It is called inside the loop that goes trough the data chunks. This way I managed to sneak in 1 000 Realm "create" transactions in between animation frames.

So the "requestAnimationFrame" was meant to be used to draw animations but could be used in this kind of scenarios and will provide a consistent behaviour. More info in the official docs: https://developer.mozilla.org/en-US/docs/Web/API/window/requestAnimationFrame

Hope this makes sense and helps someone! I'm going to close this issue now, after almost 3 years, as it doesn't seem to be an issue on Realm itself.

Hello, thanks a lot!!! I'll test this, let you know soon.

Originally posted by @cristiano-linvix in https://github.com/realm/realm-js/issues/3709#issuecomment-2072082750

stee1ix commented 2 months ago

@cristiano-linvix Did this work? I'm having the same issue of UI blocking.

sync-by-unito[bot] commented 2 months ago

➤ PM Bot commented:

Jira ticket: RJS-2850

cristiano-linvix commented 2 months ago

So, the reason for my crash was the amount of data I was receiving simultaneously.

I was using websockets to receive data from the backend, but I was receiving data from 10 different tables, and trying to record all of this at the same time.

Application would crash, I implemented better control over this, so I request table by table, save and request a new table.

This solved it for me, I hope it helps you.

I also broke ream.write every 100 records.

kneth commented 2 months ago

@stee1ix

A very large write transaction could block for an unacceptable long time, and as @cristiano-linvix suggests, breaking into smaller transactions can block for short enough time for the UI to update.