search

me0wster / javamelody

Automatically exported from code.google.com/p/javamelody
0 stars 0 forks source link

net.bull.javamelody.Counter instance taking up a lot of space #441

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
we have a module deployed on tomcat 7 in production and using javamelody 1.49.0 
to analyze performance.

our infrastructure team reported an issue in production where Javamelody's 
Counter seemed to take around 900 MB potentially due to a memory leak. is this 
a know issue. it happens after few days of usage like every other week and we 
do know what is causing this.
does anyone have an idea on this ?

Original issue reported on code.google.com by ashish.thukral321 on 17 Oct 2014 at 12:07

Attachments:

GoogleCodeExporter commented 9 years ago 
sorry for the typo in the bug statement - 
* and we do NOT know what is causing this.

Original comment by ashish.thukral321 on 17 Oct 2014 at 12:11

GoogleCodeExporter commented 9 years ago 
What is the number of http requests and of sql requests displayed in the 
statistics for the current day?

What types of http and sql requests do you have in the statistics?
Are there some dynamic values in them? Such as in "/book/show/42"?

Original comment by evernat@free.fr on 19 Oct 2014 at 4:35

GoogleCodeExporter commented 9 years ago 
If there are dynamic values in http and sql requests, you may want to define a 
javamelody parameter http-transform-pattern or sql-transform-pattern:
https://code.google.com/p/javamelody/wiki/UserGuide#6._Optional_parameters

Original comment by evernat@free.fr on 19 Oct 2014 at 4:37

GoogleCodeExporter commented 9 years ago 
we are already using the pattern for Ids and other data related patterns using 
the http transform. not monitoring any sql or jdbc etc. only HTTP calls and 
Spring method calls.

Original comment by ashish.thukral321 on 20 Oct 2014 at 5:29

GoogleCodeExporter commented 9 years ago 
Thanks for your reply.
There were 4 questions in my first comment. Unfortunately, none is answered.

Here they are now:
What is the number of http requests and of [SPRING] requests displayed in the 
statistics for the current day?
What types of http and [SPRING] requests do you have in the statistics?
Are there some dynamic values in them? Such as in "/book/show/42"?

Original comment by evernat@free.fr on 20 Oct 2014 at 8:34

GoogleCodeExporter commented 9 years ago 
ok here is some information -
1) no of different types of HTTP requests (eg /book/find /book/create) = 7
2) no of different types of Spring method calls = 31
Not sure what do you mean by What type of requests, but there are No dynamic 
data request, as we are using HTTP transform pattern. 
the issue is that the system in not under heavy load as this is our first 
release to PROD and very less number of actual users as of now, like single 
digits. with such less usage if we are getting Heap threshold warnings, it adds 
to worry how it will behave when users ramp up and hits increase.

Original comment by ashish.thukral321 on 20 Oct 2014 at 8:44

GoogleCodeExporter commented 9 years ago 
Ok, the number of http requests and of spring method calls are very low.
I can say that it is supposed to behave well and is currently used by many 
servers with thousands of different requests, as can be seen here for example:
http://javamelody.googlecode.com/svn/trunk/javamelody-core/src/site/resources/sc
reenshots/graphs.png

To go forward now, it would help if you give:
- a screenshot of the "details" of http statistics (to be sure that we speak of 
the same thing) and screenshots of the summary of http/spring/log/errors 
statistics
- the heap dump file, which you had when making your attachment above.
You can zip and upload the heap dump file on a private place and send the http 
link to the file to evernat -at- free.fr

Original comment by evernat@free.fr on 20 Oct 2014 at 10:28

GoogleCodeExporter commented 9 years ago 
I am talking to the PROD team for availability of those stats but this is the 
latest error we got - 
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOf(Arrays.java:2367)
at 
java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
at 
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.jav
a:114)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:535)
at java.lang.StringBuilder.append(StringBuilder.java:204)
at 
java.io.ObjectInputStream$BlockDataInputStream.readUTFSpan(ObjectInputStream.jav
a:3116)
at 
java.io.ObjectInputStream$BlockDataInputStream.readUTFBody(ObjectInputStream.jav
a:3024)
at 
java.io.ObjectInputStream$BlockDataInputStream.readUTF(ObjectInputStream.java:28
37)
at java.io.ObjectInputStream.readString(ObjectInputStream.java:1617)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1338)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
at 
java.util.concurrent.ConcurrentHashMap.readObject(ConcurrentHashMap.java:1557)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav
a:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1004)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1866)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1964)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1888)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1347)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:369)
at net.bull.javamelody.CounterStorage.readFromFile(CounterStorage.java:109)
at net.bull.javamelody.Counter.readFromFile(Counter.java:889)
at net.bull.javamelody.Collector.<init>(Collector.java:110)

Original comment by ashish.thukral321 on 20 Oct 2014 at 11:30

GoogleCodeExporter commented 9 years ago 
me too
I am use v1.50. after days, will occurs.

[2014/11/02 00:00:00 DEBUG masterdata] Cache Hit Ratio [masterdata]: 
0.3687242412888722
[2014/11/02 00:00:42 WARN  net.bull.javamelody] sending mail report failed
: Java heap space
[2014/11/02 00:00:59 WARN  net.bull.javamelody] sending mail report failed
java.lang.OutOfMemoryError: Java heap space

Original comment by wenhsia...@gmail.com on 2 Nov 2014 at 6:13

GoogleCodeExporter commented 9 years ago 
@ashish.thukral321
"GC overhead limit exceeded" does not give info on the cause.

As said above, it would help if you give:
- first, a screenshot of the "details" of http statistics (to be sure that we 
speak of the same thing) and screenshots of the summary of 
http/spring/log/errors statistics
- then, the heap dump file, which you had when making your attachment above.
You can zip and upload the heap dump file on a private place and send the http 
link to the file to evernat -at- free.fr

Original comment by evernat@free.fr on 13 Nov 2014 at 5:19

GoogleCodeExporter commented 9 years ago 
While waiting for info, I was wondering if the issue may be that a lot of data 
was recording when you did not have set the http-transform-pattern parameter, 
and memory is not enough to aggregate and display such data for some periods.
If yes, it's possible to remove big old data files which were written before 
setting the parameter.

Anyway, no response in the issue after a month, so resolving it as invalid / 
incomplete.

Original comment by evernat@free.fr on 9 Dec 2014 at 2:46