search

pmem / llpl

Low Level Persistence Library
Other
98 stars 38 forks source link

com.intel.pmem.llpl.HeapException: Failed to allocate memory of size #13

Open Reidddddd opened 2 years ago

Reidddddd commented 2 years ago 
com.intel.pmem.llpl.HeapException: Failed to allocate memory of size 2097152
        at com.intel.pmem.llpl.MemoryAccessor.lambda$new$0(MemoryAccessor.java:65)
        at com.intel.pmem.llpl.MemoryAccessor.<init>(MemoryAccessor.java:77)
        at com.intel.pmem.llpl.AnyMemoryBlock.<init>(AnyMemoryBlock.java:34)
        at com.intel.pmem.llpl.AbstractMemoryBlock.<init>(AbstractMemoryBlock.java:16)
        at com.intel.pmem.llpl.MemoryBlock.<init>(MemoryBlock.java:25)
        at com.intel.pmem.llpl.Heap.allocateMemoryBlock(Heap.java:226)
        at org.apache.hadoop.hbase.regionserver.PersistentMemstoreLAB.allocateBytes(PersistentMemstoreLAB.java:102)
        at org.apache.hadoop.hbase.regionserver.TestPersistentMemstoreLAB.testLABLargeAllocation(TestPersistentMemstoreLAB.java:74)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:564)

I got this exception when I tested HBase, using LLPL. The Heap size was 8MB on my server.

Use growing heap: Heap.createHeap(heapPath, heapSize);

Any hints about where went wrong?

asonje commented 2 years ago

Hello, it looks like you have run of memory on the persistent heap. if you trying to allocate multiple 2MB memory blocks then you may need to increase the heapSize value. Setting the heapSize to 8MB, for example, means the heap will not grow beyond 8MB. You will however get less than 8MB of allocatable memory. This is a characteristic of the underlying memory allocator

Reidddddd commented 2 years ago

Hello,

if you trying to allocate multiple 2MB memory blocks

I was trying allocate one 2MB block

means the heap will not grow beyond 8MB.

Emm, learning from the java docs, I was using the growing heap...

More background: I was writing a UT, and one of method was trying to test a large allocation (this issue, 2MB). And after each method, I would free(false) the MemoryBlock. So, my understanding was for each method, the Heap should contain enough memory (at least enough for 2MB), but as the exception pasted...

asonje commented 2 years ago

i wrote a small test on an 8MB growable heap does the following while (true) { mb = allocateMemoryBlock(2MB, false); mb.free(false); } This will run forever. perhaps your unit test does not always free the memory block after each test.