search

martinklepsch / boot-garden

Boot task to compile Garden stylesheets.
Eclipse Public License 1.0
37 stars 6 forks source link

Leaking memory on every compile #10

Closed egbulmer closed 8 years ago

egbulmer commented 8 years ago

Hi there! I am seeing a memory leak after every garden compile of approximately 10-15mb each time. This eventually causes an OutOfMemoryException like this one:

Exception in thread "Thread-9" java.lang.OutOfMemoryError: PermGen space
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at clojure.core_deftype__init.load(Unknown Source)
    at clojure.core_deftype__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:274)
    at clojure.lang.RT.classForName(RT.java:2154)
    at clojure.lang.RT.classForName(RT.java:2163)
    at clojure.lang.RT.loadClassForName(RT.java:2182)
    at clojure.lang.RT.load(RT.java:436)
    at clojure.lang.RT.load(RT.java:412)
    at clojure.core$load$fn__5448.invoke(core.clj:5866)
    at clojure.core$load.doInvoke(core.clj:5865)
    at clojure.lang.RestFn.invoke(RestFn.java:408)
    at clojure.core__init.load(Unknown Source)
    at clojure.core__init.<clinit>(Unknown Source)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:274)
    at clojure.lang.RT.classForName(RT.java:2154)
    at clojure.lang.RT.classForName(RT.java:2163)
    at clojure.lang.RT.loadClassForName(RT.java:2182)
    at clojure.lang.RT.load(RT.java:436)
    at clojure.lang.RT.load(RT.java:412)

I have increased my PermGen size as suggested here but this only delays the problem for a while. Since I'm making rapid changes to my styles and seeing the results reload in the browser, I very quickly run out of memory and have to re-run my Boot command, which is a pain!

I'm not sure if the leak is in the Boot task or in Garden itself. Let me know what I can do to help!

martinklepsch commented 8 years ago

Try removing the garden task from your pipeline. If you can't reproduce the problem anymore then it's most likely an issue with this task/garden.

Thanks! On Thu 24 Sep 2015 at 04:18 Elliot Bulmer notifications@github.com wrote:

Hi there! I am seeing a memory leak after every garden compile of approximately 10-15mb each time. This eventually causes an OutOfMemoryException like this one:

Exception in thread "Thread-9" java.lang.OutOfMemoryError: PermGen space at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at clojure.core_deftype__init.load(Unknown Source) at clojure.core_deftypeinit.(Unknown Source) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:274) at clojure.lang.RT.classForName(RT.java:2154) at clojure.lang.RT.classForName(RT.java:2163) at clojure.lang.RT.loadClassForName(RT.java:2182) at clojure.lang.RT.load(RT.java:436) at clojure.lang.RT.load(RT.java:412) at clojure.core$load$fn5448.invoke(core.clj:5866) at clojure.core$load.doInvoke(core.clj:5865) at clojure.lang.RestFn.invoke(RestFn.java:408) at clojure.coreinit.load(Unknown Source) at clojure.coreinit.(Unknown Source) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:274) at clojure.lang.RT.classForName(RT.java:2154) at clojure.lang.RT.classForName(RT.java:2163) at clojure.lang.RT.loadClassForName(RT.java:2182) at clojure.lang.RT.load(RT.java:436) at clojure.lang.RT.load(RT.java:412)

I have increased my PermGen size as suggested here https://github.com/boot-clj/boot/wiki/JVM-Options but this only delays the problem for a while. Since I'm making rapid changes to my styles and seeing the results reload in the browser, I very quickly run out of memory and have to re-run my Boot command, which is a pain!

I'm not sure if the leak is in the Boot task or in Garden itself. Let me know what I can do to help!

— Reply to this email directly or view it on GitHub https://github.com/martinklepsch/boot-garden/issues/10.

egbulmer commented 8 years ago

Hi @martinklepsch. I've removed it from the pipeline. The memory footprint still increases when it re-compiles the JS code, but at a much smaller rate (200-300 KB). This makes me think that either boot-garden or garden itself are the culprits.

To work around the issue I'm now generating the CSS dynamically and inserting it like this:

(require '[goog.style] '[garden.core :refer [css]]))
(goog.style/installStyles (css [:body {:background "#ff0000"}]))

I know this isn't ideal but it is fine for quick-development right now. What other info or troubleshooting can I do to assist in narrowing down the problem for you?

martinklepsch commented 8 years ago

I experimented with some different ways to use the pods but that didn't seem to have an impact on the used memory. I guess it would be useful to test with lein-garden if a similar problem exists. Other than that it might be good to remove pods completely and see how that impacts memory usage.

How are you measuring these things btw, any tools that are particularly helpful?

martinklepsch commented 8 years ago

https://github.com/boot-clj/boot/issues/268 — just discovered this issue in Boot which makes me assume it's a general issue.

egbulmer commented 8 years ago

How are you measuring these things btw, any tools that are particularly helpful?

I just used ps aux to check the memory footprint of the boot Java process. I found that whenever watch caused the style.clj to be re-compiled by boot-garden that the memory footprint increased by 8-10mb. I repeated this a number of times until I got the OutOfMemory exception.

boot-clj/boot#268 — just discovered this issue in Boot which makes me assume it's a general issue.

Interesting. I'll try lein-garden to see if it has the same problem, but it seems like it may be specific to using boot.

martinklepsch commented 8 years ago

@sonelliot Spent some time today looking into this problem. Few questions:

When the exception occurs:

  1. What are your JVM options?
  2. How much memory does the process consume?
  3. How much memory does your machine have?
egbulmer commented 8 years ago

@martinklepsch I am currently using a different machine. I will try to jump back on the other computer tomorrow and get some more detailed stats for you. In regards to your questions:

1 . What are your JVM options?

By this, do you mean the value of the BOOT_JVM_OPTIONS environment variable? Are there other system-level JVM parameters you'd like as well?

My BOOT_JVM_OPTIONS is set to:

-XX:MaxPermSize=256m -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled

2 . How much memory does the process consume?

From memory, it was approximately 1 GB before I hit the exception. I will get a more detailed measurement for you tomorrow.

3 . How much memory does your machine have?

It is a desktop PC with 12GB of DDR3 RAM.

martinklepsch commented 8 years ago

I believe this has been fixed with Boot v2.4.0. I recommend trying with a recent release of Boot.