bigdataviewer / bigdataviewer-core

ImgLib2-based viewer for registered SPIM stacks and more
BSD 2-Clause "Simplified" License
33 stars 35 forks source link

bdv running hot #112

Open tischi opened 3 years ago

tischi commented 3 years ago

@axtimwalde @StephanPreibisch @NicoKiaru

Quite regularly (but not reproducibly) I experience that the bdv uses 3 out of my 4 CPU at maximum capacity even when it seemingly just shows a "simple 2D image". I also heard this from other users.

  1. Did you ever experience this?
  2. I am bringing this up right now, because today using bdv I for the first time got this GC overhead limit error: https://www.baeldung.com/java-gc-overhead-limit-exceeded Thus, I was wondering whether maybe the high CPU usage has to do with the GC for some reason? Maybe something in the caching?
tpietzsch commented 3 years ago

@tischi

Quite regularly (but not reproducibly) I experience that the bdv uses 3 out of my 4 CPU at maximum capacity

When it happens again, could you attach VisualvM and look where it spends the time? ("Sampler" tab > "CPU")

GC overhead limit error points to "tons of very small objects that are still strongly referenced and cannot be GCed".

axtimwalde commented 3 years ago

Hi, this may be not helpful but we experienced spurious OOMs recently with the cache heavy n5-utils copy tool. The solution was to not use an outdated Oracle 8 JDK but a more recent OpenJDK. I am quoting @trautmane:

As we "learned", my runs with older 1.8 Oracle JVMs ran into memory issues likely related to older default GC approaches. Switching to openjdk-1.8.0_242 fixed those problems. Here is the VM chart for a 48 thread cap, mx 128g, openjdk-1.8.0_242 (with default GC) run on the render box

who did a few VisualVM experiments that all look 'normal' but eventually fail.

NicoKiaru commented 3 years ago

@tischi tell us the context if possible. In bdv-playground I was filling up the memory a lot and not releasing it. I fixed most of the issue I believe. If you are trying to display a simple 2D source but the jvm has only 2 MB free out of 16 Gb because of unreleased object, it will 'run hot' constantly.

Keep an eye on the memory used, explain the context, and do some visualvm inspection (thanks @tpietzsch for introducing it to me btw)

tischi commented 3 years ago

Keep an eye on the memory used, explain the context, and do some visualvm inspection

Thanks! Will do when I find a scenario where it is at least somewhat reproducible!

axtimwalde commented 3 years ago

@tischi which JDK/ JRE are you using?

tischi commented 3 years ago

@axtimwalde I was using jdk 1.8.0_161 and now, following your advise, updated to jdk 1.8.0_272. Let's see if this helps. As I said, I don't know yet how to force the issue reproducibly.