Performance between wiredTiger mongo and mmapv1

Moumouls commented 3 years ago

New Issue Checklist

[x] I am not disclosing a vulnerability.
[x] I am not just asking a question.
[x] I have searched through existing issues.
[x] I can reproduce the issue with the latest version of Parse Server.

Issue Description

Slow tests on wiredTiger, we need to check why it's so slow on this db engine.

Steps to reproduce

Run npm test with standalone mmap, time 278sec (MBP 16") Run npm test with standalone wiredTiger time 657.318 (same pc)

Actual Outcome

Slow tests

Expected Outcome

Get better or similar time

Failing Test Case / Pull Request

[ ] 🤩 I submitted a PR with a fix and a test case.
[ ] 🧐 I submitted a PR with a failing test case.

Environment

master

Server

Parse Server version: FILL_THIS_OUT
Operating system: FILL_THIS_OUT
Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): FILL_THIS_OUT

Database

System (MongoDB or Postgres): Mongo
Database version: 4.0.4 WiredTiger
Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): FILL_THIS_OUT

Client

SDK (iOS, Android, JavaScript, PHP, Unity, etc): FILL_THIS_OUT
SDK version: FILL_THIS_OUT

Logs

// MMAP
2701 specs, 0 failures, 72 pending specs
Finished in 278.188 seconds

// Wiredtiger
2701 specs, 0 failures, 72 pending specs
Finished in 657.318 seconds

dplewis commented 3 years ago

I have noticed this recently also. I haven’t had time to look into it.

Moumouls commented 3 years ago

Yes no problem, i will try to take a look next week ? @uzaysan, since you have dived deeply into performance testing are you open to help us to resolve this one ? 😃

uzaysan commented 3 years ago

@Moumouls I'm not a professional. But I will try my best. What kind of tests are we talking about?

Moumouls commented 3 years ago

@uzaysan i found a test file where the performance loss is huge and can be a good entry point here to start inspecting.

MONGODB_STORAGE_ENGINE=mmapv1 npm test spec/MongoStorageAdapter.spec.js -> 1.3sec

MONGODB_STORAGE_ENGINE=wiredTiger npm test spec/MongoStorageAdapter.spec.js -> 28.749 seconds

Good news: If the issue is at the Adapter level it will easily fixable i think

And also in the open source world, professional or not, it doesn't matter, contributing and helping is always welcome and will always be a step forward :)

dplewis commented 3 years ago

@Moumouls I reduced the testing time in my branch

https://github.com/parse-community/parse-server/tree/improve-schema-cache

The reason why MongoStorageAdapter.spec.js is slow is because its over deleting the database between tests. The over deleting the entire database between tests also occurs in other test specs files.

I have a few ideas for improving test speeds.

Moumouls commented 3 years ago

@dplewis what time difference do you have between mma and wired ?

Moumouls commented 3 years ago

You are right @dplewis, delete calls on wiredtiger seems to be super slow

I have modified before with

 beforeEach(done => {
    console.time('perf');
    new MongoStorageAdapter({ uri: databaseURI }).deleteAllClasses().then(() => {
      console.timeEnd('perf');
      done();
    }, fail);
  });

MMA log

Started
perf: 8.118ms
.perf: 8.118ms
.perf: 6.268ms
.perf: 6.516ms
.perf: 10.218ms
.perf: 7.256ms
.perf: 10.502ms
.perf: 7.048ms
.perf: 7.569ms
.perf: 6.7ms
.perf: 7.362ms
.perf: 7.585ms
.perf: 6.12ms
.perf: 6.409ms
.perf: 5.209ms
.perf: 5.952ms
.

Wired Log

perf: 389.629ms
.perf: 374.266ms
.perf: 378.7ms
.perf: 381.291ms
.perf: 426.412ms
.perf: 442.377ms
.perf: 415.312ms
.perf: 412.833ms
.perf: 395.817ms
.perf: 437.409ms
.perf: 452.292ms
.perf: 428.49ms
.perf: 430.66ms
.perf: 380.998ms
.perf: 456.02ms
.perf: 431.247ms
.

Moumouls commented 3 years ago

A simple create find seems super slow too

fit('find succeeds when query is within maxTimeMS', done => {
    const maxTimeMS = 250;
    const adapter = new MongoStorageAdapter({
      uri: databaseURI,
      mongoOptions: { maxTimeMS },
    });
    console.time('perf');
    adapter
      .createObject('Foo', { fields: {} }, { objectId: 'abcde' })
      .then(() => adapter._rawFind('Foo', { $where: `sleep(${maxTimeMS / 2})` }))
      .then(
        () => {
          console.timeEnd('perf');
          done();
        },
        err => {
          done.fail(`maxTimeMS should not affect fast queries ${err}`);
        }
      );
  });

MMA : 158.848ms Wired: 335.838ms

Only testing createObject: MMA: 4.269ms Wired: 149.538ms

Only testing rawFind: MMA: 157.602ms Wired: 155.504ms

So class create/deletion seems to have issue

dplewis commented 3 years ago

@Moumouls I recently improved the JS SDK test suite and I found that if you use beforeEach in the helper.js and in your specs the tests run slower or they may run out of sync. Can you try changing beforeEach to beforeAll in the helper.js file and run your perf tests?

https://github.com/parse-community/Parse-SDK-JS/pull/1292/files#diff-d20c72e61c31d8c8f138c8d2d0966d5a24a99959227d55edcd105fca781529c0R118

I used your example and got for wired.

perf: 460.846ms
.perf: 6.841ms
.perf: 3.854ms
.perf: 3.453ms
.perf: 54.001ms
.perf: 56.075ms
.perf: 54.925ms
.perf: 55.055ms
.perf: 55.091ms
.perf: 54.957ms
.perf: 55.085ms
.perf: 55.478ms
.perf: 55.626ms
.perf: 6.603ms
.perf: 183.378ms
.perf: 129.419ms

Moumouls commented 3 years ago

Thanks for the investigation @dplewis, I'll try to take a look asap :)

parse-community / parse-server