search

Kinetic / kinetic-java

Kinetic Java Library and Simulator.
25 stars 17 forks source link

java.lang.OutOfMemoryError: Java heap space #3

Open toolslive opened 10 years ago

toolslive commented 10 years ago

While using the C++ client to fetch a keyrange, I get this exception:

Exception in thread "Simulator-16" java.lang.OutOfMemoryError: Java heap space
    at com.google.protobuf.ByteString.copyFrom(ByteString.java:192)
    at com.google.protobuf.CodedInputStream.readBytes(CodedInputStream.java:324)
    at com.seagate.kinetic.proto.KineticDb$Versioned.<init>(KineticDb.java:131)
    at com.seagate.kinetic.proto.KineticDb$Versioned.<init>(KineticDb.java:68)
    at com.seagate.kinetic.proto.KineticDb$Versioned$1.parsePartialFrom(KineticDb.java:164)
    at com.seagate.kinetic.proto.KineticDb$Versioned$1.parsePartialFrom(KineticDb.java:159)
    at com.seagate.kinetic.proto.KineticDb$Versioned$Builder.mergeFrom(KineticDb.java:1304)
    at com.seagate.kinetic.proto.KineticDb$Versioned$Builder.mergeFrom(KineticDb.java:1184)
    at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337)
    at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
    at com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:170)
    at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:882)
    at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267)
    at com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:161)
    at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:875)
    at com.seagate.kinetic.simulator.persist.KVValue.<init>(KVValue.java:35)
    at com.seagate.kinetic.simulator.persist.leveldb.LevelDbStore.getRange(LevelDbStore.java:395)
    at com.seagate.kinetic.simulator.persist.leveldb.LevelDbStore.getRange(LevelDbStore.java:59)
    at com.seagate.kinetic.simulator.persist.RangeOp.operation(RangeOp.java:125)
    at com.seagate.kinetic.simulator.internal.SimulatorEngine.processRequest(SimulatorEngine.java:419)
    at com.seagate.kinetic.simulator.io.provider.nio.NioQueuedRequestProcessRunner.doProcessMessage(NioQueuedRequestProcessRunner.java:190)
    at com.seagate.kinetic.simulator.io.provider.nio.NioQueuedRequestProcessRunner.run(NioQueuedRequestProcessRunner.java:136)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)

Basically, I'm doing this:

void dump(std::shared_ptr<kinetic::BlockingKineticConnection> bc){
    std:: string last_key(4096, '\xff');
    int count = 0;
    for (kinetic::KeyRangeIterator it =
             bc -> IterateKeyRange("", true, last_key, true, 1000);
         it != kinetic::KeyRangeEnd();
         ++it) {
        printf("%s\n", it->c_str());
        count ++;
    }
    std:: cout << std::endl << "count=" << count << std::endl;
}
jphughes commented 10 years ago

Yes. Define VM argument like -Xmx1g should fix the problem.

Seems like you are not running the latest version of kinetic-java. Do a pull update may also help.

Sincerely

Jim

On Aug 19, 2014, at 1:02 AM, Romain Slootmaekers notifications@github.com wrote:

While using the C++ client to fetch a keyrange, I get this exception:

Exception in thread "Simulator-16" java.lang.OutOfMemoryError: Java heap space at com.google.protobuf.ByteString.copyFrom(ByteString.java:192) at com.google.protobuf.CodedInputStream.readBytes(CodedInputStream.java:324) at com.seagate.kinetic.proto.KineticDb$Versioned.(KineticDb.java:131) at com.seagate.kinetic.proto.KineticDb$Versioned.(KineticDb.java:68) at com.seagate.kinetic.proto.KineticDb$Versioned$1.parsePartialFrom(KineticDb.java:164) at com.seagate.kinetic.proto.KineticDb$Versioned$1.parsePartialFrom(KineticDb.java:159) at com.seagate.kinetic.proto.KineticDb$Versioned$Builder.mergeFrom(KineticDb.java:1304) at com.seagate.kinetic.proto.KineticDb$Versioned$Builder.mergeFrom(KineticDb.java:1184) at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:337) at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267) at com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:170) at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:882) at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:267) at com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:161) at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:875) at com.seagate.kinetic.simulator.persist.KVValue.(KVValue.java:35) at com.seagate.kinetic.simulator.persist.leveldb.LevelDbStore.getRange(LevelDbStore.java:395) at com.seagate.kinetic.simulator.persist.leveldb.LevelDbStore.getRange(LevelDbStore.java:59) at com.seagate.kinetic.simulator.persist.RangeOp.operation(RangeOp.java:125) at com.seagate.kinetic.simulator.internal.SimulatorEngine.processRequest(SimulatorEngine.java:419) at com.seagate.kinetic.simulator.io.provider.nio.NioQueuedRequestProcessRunner.doProcessMessage(NioQueuedRequestProcessRunner.java:190) at com.seagate.kinetic.simulator.io.provider.nio.NioQueuedRequestProcessRunner.run(NioQueuedRequestProcessRunner.java:136) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744)

Basically, I'm doing this:

void dump(std::shared_ptrkinetic::BlockingKineticConnection bc){ std:: string last_key; for(int i =0; i < 4096; ++i){ last_key += "\xff"; } int count = 0; for (kinetic::KeyRangeIterator it = bc -> IterateKeyRange("", true, last_key, true, 1000); it != kinetic::KeyRangeEnd(); ++it) { printf("%s\n", it->c_str()); count ++; } std:: cout << std::endl << "count=" << count << std::endl; }

— Reply to this email directly or view it on GitHub.

toolslive commented 10 years ago

the most recent version of the simulator uses protocol version 2.0.6 while the most recent version of the cpp client is using protocol version 2.0.4 they don't seem to like each other.

jphughes commented 10 years ago

There will be a series of pushes later this week that will bring all the clients and simulator up to version 3.0.0.

Sincerely

Jim

On Aug 19, 2014, at 3:02 PM, Romain Slootmaekers notifications@github.com wrote:

the most recent version of the simulator uses protocol version 2.0.6 while the most recent version of the cpp client is using protocol version 2.0.4 they don't seem to like each other.

— Reply to this email directly or view it on GitHub.