retained size of objects issue

[jackmalkovick]

I'm trying to use your nice library in order to compute the retained size of an object, because I have a huge hprof file.

I used HeapFactory.createHeap(), because I see that getRetainedSize() is not implemeted for FastHprofHeap.

It works, the only thing is that it gives a different result compared with Yourkit or Eclipse Memory Analyzer (those two give the exact same number of bytes each time).

Example

                Map<Integer, String> map = new HashMap<>();
        for (int i = 0; i < numberOfItems; i++) {
            map.put(i, Integer.toString(i));
        }

The bigger the numberOfItems, the bigger the difference in retained size between this api and the standard tools. E.g for 5 items, this library returns 730 and Yourkit and EMA return 544 for 10_000 items, this library returns 1,366,404 and Yourkit and EMA return 1,023,568

Any ideas? Thank you!

[jackmalkovick]

Extra information. E.g. 0 -> "0" map entry

The API reports retained size (in bytes) like this

org.netbeans.lib.profiler.heap.InstanceDump / java.util.HashMap$Node = 98
   org.netbeans.lib.profiler.heap.InstanceDump / java.lang.Integer = 20
   org.netbeans.lib.profiler.heap.InstanceDump / java.lang.String = 54
      org.netbeans.lib.profiler.heap.PrimitiveArrayDump / char[] = 26

Yourkit / EMA tools like this

[0]  java.util.HashMap$Node (0 → "0") chunk size = 80 
      value  java.lang.String "0" 48 
            value  char[1] "0" 24
      key  java.lang.Integer = 0 16 

For Integer at least, shouldn't be the size indeed 16 bytes? It's my understanding that we have a 12-byte object header, padded to a multiple of 8 bytes, so 16 bytes?

[aragozin]

Let me clarify. Code of this library is taken from NetBeans profiler (which is also powering Visual VM). My changes were focused for dealing with huge (100+ GiB) heaps. Retained size is questionable metric (for my purposes at least) so I never used it.

I have taken a quick look. I can say what code assume header to be 16 bytes (which would be true for 32+ GiB heap) and completely disregard alignment.

I suggest you check Visual VM behavior. If Visual VM is showing better numbers, I probably, should merge up stream fixes. If it is the same, you could reach Visual VM forum.

[jackmalkovick]

You are right! Thank you

[wind57]

@jackmalkovick may I ask why you would not use jol?

[aragozin]

@wind57 to my knowledge jol does not work with heap dumps

[wind57]

@alexey-ragozin correct. https://www.youtube.com/watch?v=KIGcmLL67CQ in russian, but still the name of the video should tell @jackmalkovick not to do that with heap dumps.