pubkey / rxdb

A fast, local first, reactive Database for JavaScript Applications https://rxdb.info/
https://rxdb.info/
Apache License 2.0
21.56k stars 1.06k forks source link

Slow event loop? when attempting to sync with an offline server #601

Closed gvuyk closed 6 years ago

gvuyk commented 6 years ago

Case

Bug

Issue

I have 12 collections that need a 1 time (non-live) sync, so I pass a PouchDb instance as a 'remote' to each collection's sync() method. Everything works as expected except when the remote server is offline, then its like if the event loop is processed reaally slowly for a few minutes until maybe a timeout kicks in and things go back to normal. No error is reported by the RxReplicationStates

Will try to provide a test case after the weekend

Info

Code

const RxDB = require("rxdb");
const PouchDB = require("pouchdb");
const fs = require("fs");

RxDB.plugin(require("pouchdb-adapter-memory"));

async function startReplication()
{
    var db = await RxDB.create({
        name: "testdb",
        adapter: "memory",
        multiInstance: false
    });
    var mySchema = {
        version: 0,
        type: "object",
        properties: {
            passportId: {type: "string"}
        }
    };
    for(let i=0; i<12; i++)
    {
        let collection = await db.collection({name: "collection"+i, schema: mySchema});
        collection.sync({remote: new PouchDB("http://offlineserver/collection"+i)});
    }
}

startReplication().then(() => {
    console.time("fileRead");
    fs.readFile("file", (error, data) => {    // file doesn't need to exist
        console.timeEnd("fileRead");
    });
});
pubkey commented 6 years ago

It's not clear to me what you mean with event loop is processed reaally slowly. Have you done a chrome-profiling?

gvuyk commented 6 years ago

No, sorry, did the profiling now and the event loop seems ok. Its the thread pool that gets blocked. If the amount of collections to sync is <= 4 (the default pool size) the problem goes unnoticed, increasing it to something equal or greater than the amount of collections (setting UV_THREADPOOL_SIZE env var to 12) has the same effect

Updated the code to reproduce it

pubkey commented 6 years ago

Have you observed the network? I think this is a pouchdb-thing which checks in 12 loops if the server is now reachable. Can you set retry to false and check if the problem still occurs?

pubkey commented 6 years ago

I'm closing this because it's a pouchdb/leveldb-problem which should be resolved at their repo.

gvuyk commented 6 years ago

It turned out to be a dns module problem, issue and workarounds