search

AnantLabs / google-enterprise-connector-manager

Automatically exported from code.google.com/p/google-enterprise-connector-manager
Apache License 2.0
0 stars 1 forks source link

OutOfMemoryError submitting feed. #207

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
A customer complained that continuous feeds caused his Windows Virtual Machine 
to run out of 
VM.  Some speculation indicated that the GSA had become slow to respond.  When 
running over 
our VPN (slow connection to GSA) I believe I was able to recreate this.

Dec 19, 2009 2:08:21 PM [Traverse flmobius] 
com.google.enterprise.connector.traversal.QueryTraverser runBatch
SEVERE: Push Exception during traversal.
com.google.enterprise.connector.pusher.PushException: Unable to allocate feed 
buffer.  Try 
reducing the maxFeedSize setting.: Java heap space
  at com.google.enterprise.connector.pusher.DocPusher.take(DocPusher.java:225)
  at com.google.enterprise.connector.traversal.QueryTraverser.runBatch(QueryTraverser.java:189)
  at com.google.enterprise.connector.instantiator.CancelableBatch.run(CancelableBatch.java:72)
  at 
com.google.enterprise.connector.instantiator.ThreadPool$CancelTimeoutRunnable.ru
n(ThreadPo
ol.java:239)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
  at java.util.concurrent.FutureTask.run(FutureTask.java:138)
  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  at java.lang.Thread.run(Thread.java:637)
Caused by: java.lang.OutOfMemoryError: Java heap space
  at java.io.ByteArrayOutputStream.<init>(ByteArrayOutputStream.java:60)
  at com.google.enterprise.connector.pusher.XmlFeed.<init>(XmlFeed.java:95)
  at com.google.enterprise.connector.pusher.DocPusher.take(DocPusher.java:223)
  ... 15 more

If the GSA is slow to accept feeds, then too many feeds may be queued up 
waiting to be sent, 
each with a 10MB buffer of data.   Memory allocation grows until it is 
exhausted.

In my FileList connector, I was sending a new feed every 4 seconds. I can 
imagine that could 
overwhelm the GSA.

Original issue reported on code.google.com by Brett.Mi...@gmail.com on 19 Dec 2009 at 10:42

GoogleCodeExporter commented 9 years ago

Original comment by Brett.Mi...@gmail.com on 21 Dec 2009 at 5:36

GoogleCodeExporter commented 9 years ago

Original comment by jla...@google.com on 8 Jan 2010 at 10:47

GoogleCodeExporter commented 9 years ago 
Fixed 11 January 2010 in Connector Manager revision r2404

Original comment by Brett.Mi...@gmail.com on 12 Jan 2010 at 1:17

GoogleCodeExporter commented 9 years ago

Original comment by jla...@google.com on 19 Apr 2010 at 9:54