NoSQL DBMS Evaluation

[ppyordanov]

A NoSQL database management system is going to be evaluated based on a number of different criteria. Results will be compared to the performance of MySQL DBMS.

SQL Evaluation details: #29

DBMS choice information: #27

[ppyordanov]

• MongoDB has been chosen for evaluation: http://www.mongodb.org/downloads
• manual: http://docs.mongodb.org/manual/reference/command/

[ppyordanov]

__Benchmark Results__

Four benchmarks were run with different data set sizes to assess the following factors:

• insertion
• selection
• deletion
• update

Each test was completed twice /no caching/ to improve accuracy.

_TEST 1: 5000_

first run: second run:

Test MONGO_DB
MongoDB

DATA: skipped

RESULT:

Insert single record:
Single insert in collections: Execution time: 0 min, 0 sec, 10 millisec

Insert real data (SIM):
Execution time: 0 min, 1 sec, 656 millisec Data size:
1 * 5000

Update single record:
Execution time: 0 min, 0 sec, 9 millisec

Get single record:
Execution time: 0 min, 0 sec, 33 millisec

Delete single record:
Execution time: 0 min, 0 sec, 0 millisec

Get all records:
Execution time: 0 min, 0 sec, 518 millisec

TOTAL:

Execution time: 0 min, 2 sec, 226 millisec

STATS:

> db.DataReadings.stats()
{
        "ns" : "ugmap.DataReadings",
        "count" : 4999,
        "size" : 1199760,
        "avgObjSize" : 240,
        "storageSize" : 2793472,
        "numExtents" : 5,
        "nindexes" : 1,
        "lastExtentSize" : 2097152,
        "paddingFactor" : 1,
        "systemFlags" : 1,
        "userFlags" : 1,
        "totalIndexSize" : 171696,
        "indexSizes" : {
                "_id_" : 171696
        },
        "ok" : 1
}

[ppyordanov]

_TEST 2: 60000_

_first run:_

_second run:_

Test MONGO_DB
MongoDB

DATA: skipped

RESULT:

Insert single record:
Single insert in collections: Execution time: 0 min, 0 sec, 11 millisec

Insert real data (SIM):
Execution time: 0 min, 6 sec, 126 millisec Data size:
12 * 5000

Update single record:
Execution time: 0 min, 0 sec, 4 millisec

Get single record:
Execution time: 0 min, 0 sec, 19 millisec

Delete single record:
Execution time: 0 min, 0 sec, 5 millisec

Get all records:
Execution time: 0 min, 1 sec, 728 millisec

TOTAL:

Execution time: 0 min, 7 sec, 893 millisec

STATS:

> db.DataReadings.stats()
{
        "ns" : "ugmap.DataReadings",
        "count" : 59999,
        "size" : 14399760,
        "avgObjSize" : 240,
        "storageSize" : 22507520,
        "numExtents" : 7,
        "nindexes" : 1,
        "lastExtentSize" : 11325440,
        "paddingFactor" : 1,
        "systemFlags" : 1,
        "userFlags" : 1,
        "totalIndexSize" : 1954064,
        "indexSizes" : {
                "_id_" : 1954064
        },
        "ok" : 1
}

[ppyordanov]

_TEST 3: 200000_

_first run:_

_second run:_

Test MONGO_DB
MongoDB

DATA: skipped

RESULT:

Insert single record:
Single insert in collections: Execution time: 0 min, 0 sec, 11 millisec

Insert real data (SIM):
Execution time: 0 min, 14 sec, 673 millisec Data size:
40 * 5000

Update single record:
Execution time: 0 min, 0 sec, 4 millisec

Get single record:
Execution time: 0 min, 0 sec, 19 millisec

Delete single record:
Execution time: 0 min, 0 sec, 3 millisec

Get all records:
Execution time: 0 min, 4 sec, 212 millisec

TOTAL:

Execution time: 0 min, 18 sec, 923 millisec

STATS:

> db.DataReadings.stats()
{
        "ns" : "ugmap.DataReadings",
        "count" : 199999,
        "size" : 47999760,
        "avgObjSize" : 240,
        "storageSize" : 58441728,
        "numExtents" : 9,
        "nindexes" : 1,
        "lastExtentSize" : 20643840,
        "paddingFactor" : 1,
        "systemFlags" : 1,
        "userFlags" : 1,
        "totalIndexSize" : 6508096,
        "indexSizes" : {
                "_id_" : 6508096
        },
        "ok" : 1
}

[ppyordanov]

One more test was done, with 5m insertions:

_TEST 4: 5m._

Test MONGO_DB
MongoDB

DATA: skipped

RESULT:

Insert single record:
Single insert in collections: Execution time: 0 min, 0 sec, 8 millisec

Insert real data (SIM):
Execution time: 5 min, 4 sec, 833 millisec Data size:
1000 * 5000

Update single record:
Execution time: 0 min, 0 sec, 4 millisec

Get single record:
Execution time: 0 min, 0 sec, 20 millisec

Delete single record:
Execution time: 0 min, 0 sec, 4 millisec

Get all records:
Execution time: 3 min, 24 sec, 669 millisec

TOTAL:

Execution time: 8 min, 29 sec, 538 millisec

STATS:

> db.DataReadings.stats()
{
        "ns" : "ugmap.DataReadings",
        "count" : 4999999,
        "size" : 1199999760,
        "avgObjSize" : 240,
        "storageSize" : 1580150784,
        "numExtents" : 19,
        "nindexes" : 1,
        "lastExtentSize" : 415174656,
        "paddingFactor" : 1,
        "systemFlags" : 1,
        "userFlags" : 1,
        "totalIndexSize" : 162228192,
        "indexSizes" : {
                "_id_" : 162228192
        },
        "ok" : 1
}

[ppyordanov]

Comment moved #27

[ppyordanov]

_Added Mongo DB benchmarks + results to repository: 350dd9f_