OnestoreEntity memory leak

[GoogleCodeExporter]

Running queries and seeing memory consumption go way up due to these 
classes not being GC'ed...

com.google.storage.onestore.v3.OnestoreEntity$PropertyValue 66,840,592
com.google.storage.onestore.v3.OnestoreEntity$Property          47,743,240

The number to the right is the heap size in memory.  Memory looks okay in 
com.googlecode.objectify, and in my package.  Scoping the app w/ JProbe.  

Perhaps it's due to DB connections not being closed, found this...

http://groups.google.com/group/google-appengine-
java/browse_thread/thread/afc98d9a853601d2

Wondering about the objectify connection session, and the lack of a close() 
call.  Perhaps the wiki could contain some info about this?  

Gonna need to get to the bottom of this soon, as my app can only run for a 
couple hours before running out of memory. 

Objectify v2.2. GAE v1.3.4.

Original issue reported on code.google.com by licht...@gmail.com on 23 May 2010 at 9:17

[GoogleCodeExporter]

Retrieved this stack trace...

com.google.storage.onestore.v3.OnestoreEntity$Property.<init>() OnestoreEntity.j
ava:2659
com.google.storage.onestore.v3.OnestoreEntity$EntityProto.addRawProperty()
OnestoreEntity.java:5434
com.google.storage.onestore.v3.OnestoreEntity$EntityProto.mergeFrom(com.google.s
torage.onestore.v3.OnestoreEntity$EntityProto)
OnestoreEntity.java:5511
com.google.storage.onestore.v3.OnestoreEntity$EntityProto.mergeFrom(com.google.a
ppengine.repackaged.com.google.io.protocol.ProtocolMessage)
OnestoreEntity.java:5192
com.google.appengine.repackaged.com.google.io.protocol.ProtocolMessage.clone()
ProtocolMessage.java:220
com.google.appengine.api.datastore.dev.LocalDatastoreService.putImpl(com.google.
appengine.tools.development.LocalRpcService$Status,
com.google.apphosting.api.DatastorePb$PutRequest)   LocalDatastoreService.java:453
com.google.appengine.api.datastore.dev.LocalDatastoreService.put(com.google.appe
ngine.tools.development.LocalRpcService$Status,
com.google.apphosting.api.DatastorePb$PutRequest)   LocalDatastoreService.java:438
sun.reflect.GeneratedMethodAccessor3.invoke(java.lang.Object, java.lang.Object 
[ ])
Not Available
sun.reflect.DelegatingMethodAccessorImpl.invoke(java.lang.Object, 
java.lang.Object
[ ])    Not Available
java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object [ ]) Not 
Available
com.google.appengine.tools.development.ApiProxyLocalImpl$AsyncApiCall.call()
ApiProxyLocalImpl.java:307
com.google.appengine.tools.development.ApiProxyLocalImpl$AsyncApiCall.call()
ApiProxyLocalImpl.java:264
java.util.concurrent.Executors$PrivilegedCallable$1.run()   Not Available
java.security.AccessController.doPrivileged(java.security.PrivilegedAction,
java.security.AccessControlContext) Not Available
java.util.concurrent.Executors$PrivilegedCallable.call()    Not Available
java.util.concurrent.FutureTask$Sync.innerRun() Not Available
java.util.concurrent.FutureTask.run()   Not Available
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(java.lang.Runnable)  Not 
Available
java.util.concurrent.ThreadPoolExecutor$Worker.run()    Not Available
java.lang.Thread.run()  Not Available

Original comment by licht...@gmail.com on 23 May 2010 at 10:43

[GoogleCodeExporter]

It's *extraordiarily* unlikely that this is an issue with Objectify.  Objectify 
has no close() method because (like 
the low-level API) it maintains no session state.  There's nothing to close.

If you want to post this information to the Objectify mailing list, we might be 
able to help you diagnose the 
issue.  Some more information that might be helpful:  This is for the local 
environment, right?  Do you see any 
similar effect on the production GAE?  You'll see periodic "your application 
has crossed the soft memory limit" 
messages and your app will restart as a new instance.

Are you creating one or more new Objectify instances (which create new GAE 
DatastoreService objects) for 
every request and then abandoning them at the end of the request?  Or are you 
sharing Objectify instances 
across threads (a documented no-no)?

It looks like a GAE datastore issue.  Probably you will need to open up an 
issue in the GAE issue tracker.  But 
we can at least help you isolate it on the mailing list.

Original comment by lhori...@gmail.com on 24 May 2010 at 7:44

• Changed state: Invalid

[GoogleCodeExporter]

Right, it's definitely a GAE datastore issue in the local dev server.  On 
production
seeing restarts every 20~60 minutes, so this issue is probably a mute point on
production.  

There's no close, that's a given, but don't see any docs explaining why.  
Perhaps
that's another issue that should be entered to have some explanation.  

All servlets create their own DAO following the best practices approach.  There 
are
some cases where I jump from one Servlet to another; query takes place in one 
servlet
w/ a DAO, object is passed to another servlet and then put using its DAO.  This
probably violates using Objectify instances across threads.  Perhaps this my 
memory leak?

Original comment by licht...@gmail.com on 24 May 2010 at 9:10

[GoogleCodeExporter]

I can imagine a document which describes "how Objectify is different from JDO" 
which includes a discussion of 
why JDO has a close() method and why Objectify does not... but if you're 
puzzled by this, you should be puzzled 
why a close() method is necessary, not the other way 'round!

Are you keeping the Objectify/DAO instance as a member of your Servlet, and 
thus sharing it among all the 
threads that access the servlet?  I suspect I know what your problem is, but 
the bug database is not an 
appropriate place to be having this discussion.  Please continue here: 
http://groups.google.com/group/objectify-appengine

Original comment by lhori...@gmail.com on 24 May 2010 at 9:54

[GoogleCodeExporter]

Half a dozen one or the other.  

Keeping track of the session in a large scale project can be problematic.  

Original comment by licht...@gmail.com on 24 May 2010 at 10:10

[GoogleCodeExporter]

I have same issue. GAE SDK 1.6.1 and Objectify 3.1 under local development 
server.

Original comment by rus...@walkmind.com on 12 Jan 2012 at 3:42

[GoogleCodeExporter]

This is almost certainly *not* an issue with Objectify.  If you would like 
assistance, please try asking on the mailing list.

Original comment by lhori...@gmail.com on 12 Jan 2012 at 3:56

[GoogleCodeExporter]

For future visitors of this issue who come from Google search results (as I 
did) I want to say that it's behavior by design for Java version of GAE SDK.

http://groups.google.com/group/google-appengine-java/browse_thread/thread/e8c5d4
947bc61593?pli=1

Original comment by rus...@walkmind.com on 13 Jan 2012 at 12:07