OrientDb don't release memory

[hosseinGanjyar]

I do to insert about ~8000000 records into my OrientDb, But I have got one problem that java's memory allocated increase once inserting. Therefore I want to release this memory after insert every record.

#My Code:

var dbConn = [];
var dbNext = 0;
var dbMax = 25;

for (var i = 0; i <= dbMax; i++) {
    var db = new ODatabase({
        host: orientdb.host,
        port: 2424,
        username: 'root',
        password: orientdb.password,
        name: 'test',
    });
    dbConn.push(db);
}
//Start loop
record["@class"] = "table";
var db = nextDB();
db.open().then(function () {
      return db.record.create(record);
}).then(function (res) {
      db.close().then(function () {
               //----resume loop
    });
    }).error(function (err) {
            //------
    });

function nextDB() {
    if (++dbNext >= dbMax) {
        dbNext -= dbMax;
    }
    return dbConn[dbNext];
}

memory befor start: 1.2g memory after 4hours: 4.7g

• OrientDB Version [latest]
• NodeJS
• Windows 10

[andrii0lomakin]

@hosseinGanjyar Java does not have a method to release direct memory once it is allocated. You may only limit usage of direct memory but do not release it.

[hosseinGanjyar]

Ok, But when I select / insert into OrientDb database, increase memory. What to do release this memory after select / insert into OrientDb?

[andrii0lomakin]

@hosseinGanjyar Java can not release memory, we can not do it programmatically. We can only limit its usage.

[andrii0lomakin]

Hi @hosseinGanjyar after internal discussion we decided that we will implement clean of memory using proprietary Oracle API. But please note that in such case requests processing will be slow and eventually DB will consume the same amount of memory. Is it OK for you ?

[andrii0lomakin]

Hi @hosseinGanjyar what do you think about such approach ?

[hosseinGanjyar]

@laa I think using OrientJS isn't good for data transport, and OrientDB is very stupid because haven't streaming. I has problem with Massive Insertion using NodeJS into OrientDB that take very much times of me, and I didn't get any result yet.

[wolf4ood]

@hosseinGanjyar i've already sent you the code.

What do you mean by OrientJS isn't good for data transport? What do you mean by does not support streaming?

I've written for you a code that import 8000000~ in more/less 25 min

[saeedtabrizi]

@hosseinGanjyar i'm not agree . orientdb is a modern distributed nosql multimodel database and is smart too . but if don't know the concepts of using NOSQL or orientdb please read the documentation more and more . i suggest , if you want to have more control to memory management use the orientdb java api .

[andrii0lomakin]

Hi, we need to understand , whether this feature needs to be implemented. It requires a lot of coding so if it is not needed I prefer rather to close it.

[tglman]

Hi,

Looking at this specific discussion i don't see enough information to start any implementation, there is just a generic blaming of high memory usage, so if you don't have any other profiling/measuring that show that the issue is the closing of byte buffers i wouldn't suggest to start any huge coding task, especially based on this issue.

This issue should instead trigger a profiling of the general memory usage of the whole system to figured out if the issue is from disc cache or high memory retained on heap, or just "normal lazyness" on releasing memory of java after allocation of a lot of objecs.

Hope this help.

Regards

[andrii0lomakin]

Close issue because of no reply. If you reproduce given issue and have information which is requested in issue feel free to reopen it.

[hosseinGanjyar]

@tglman see this issue: https://github.com/orientechnologies/orientdb/issues/7090