Memory leak?

[rogerallen]

I've written a library using clisk (http://github.com/rogerallen/tweegeemee) and it suffers from some slow memory leaks that I think I've traced to clisk.

To reproduce, what I did was create a new app with lein new testclisk and added [net.mikera/clisk "0.10.0"] to the :dependencies. Then in one terminal, do lein repl. In another open jvisualvm.

In jvisualvm open the clojure app in the Monitor pane watch the Metaspace memory graph.

In the repl, do:

(use 'clisk.live)
(dotimes [i 200] (image pos :size 16))

After you run that last line, press the "Perform GC" button and what I see is a sharp jump in the Used Metaspace. About 9M each time I run that command.

;; starting at Used: 109,951,136 B
(dotimes [i 200] (image pos :size 16))
;; GC, then I see: 118,946,072 B
(dotimes [i 200] (image pos :size 16))
;; GC, then I see: 127,942,064 B

I don't know how to get that memory back. Can you advise? Thanks.

[mikera]

I've seen the project very cool ! :-)

Clisk expends and compiles code for performance reasons, so it will be creating new class files that go in metaspace. I'm not sufficiently familiar with the GC behaviour on class files, but I suspect this might be what is happening?

Do you ever get an OOM exception?

[mikera]

Sorry just looked at your other issue. I think you may need to limit the JVM size, so that it forces a GC rather than blowing heroku limits?

[rogerallen]

I'll see what happens if I add -XX:MaxMetaspaceSize=100m to force a metaspace GC. I'm not that familiar with memory problems, either...

[rogerallen]

Adding to the project.clj

:jvm-opts     ^:replace ["-Xms128m" "-Xmx400m" "-Xss512k" "-XX:MaxMetaspaceSize=100m"]

Then I do get an out-of-memory error pretty quickly:

user=> (use 'clisk.live)
nil
user=> (dotimes [i 200] (image pos :size 16))
nil
user=> (dotimes [i 200] (image pos :size 16))
CompilerException java.lang.OutOfMemoryError: Metaspace, compiling:(/private/var/folders/zb/w97rn0bn1hl7stfrgk5_04mc0000gn/T/form-init5546680249120895064.clj:1:1) 

Here's the full project.clj

(defproject testclisk "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.6.0"]
                 [net.mikera/clisk    "0.10.0"]]
  :jvm-opts     ^:replace ["-Xms128m" "-Xmx400m" "-Xss512k" "-XX:MaxMetaspaceSize=100m"]
  )

[mikera]

Hmmm I can't replicate the issue even if I run it 20,000 times. Though it does seem very slow.

[mikera]

What is OS / JVM combination?

[rogerallen]

Shoot. I was hoping this would repro easily. The issue builds up slowly over time on Heroku, but I'm testing on a Mac OS X 10.9.5.

> uname -a
Darwin thunder.local 13.4.0 Darwin Kernel Version 13.4.0: Wed Mar 18 16:20:14 PDT 2015; root:xnu-2422.115.14~1/RELEASE_X86_64 x86_64
> lein version
Leiningen 2.5.1 on Java 1.8.0_45 Java HotSpot(TM) 64-Bit Server VM

[rogerallen]

I added the full project.clj to the comment above, just in case it helps.

[rogerallen]

I can't reproduce either--unless I am watching with jvisualvm. (grrr...)

I think constraining my own app's metaspace is what I need to do. Sorry for the errant issue. I'll close this now.