search

PILLUTLAAVINASH / google-enterprise-connector-manager

Automatically exported from code.google.com/p/google-enterprise-connector-manager
0 stars 0 forks source link

Enable the WorkQueueItem.timeout to be configurable #90

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
Current WorkQueueItem.timeout is 'static final' and defaults to 5 minutes.
 This means any work item that takes more 5 minutes will be interrupted.

Some connector implementations are having a problem with this since the
thread interrupted state is checked after the DocumentList is constructed
but before it is sent to the DocPusher.  If the startTraversal() or
resumeTraversal() takes more time than the WorkQueueItem.timeout() the
InterrupterThread will interrupt the WorkQueueThread.

It might be nice to allow the WorkQueueItem.timeout value to be set during
the construction of the WorkQueue so it can be configured using the
applicationContext.xml file.

Original issue reported on code.google.com by mgron...@gmail.com on 12 May 2008 at 6:27

GoogleCodeExporter commented 8 years ago 
r794 | mgronber | 2008-05-16 14:35:06 -0700 (Fri, 16 May 2008) | 22 lines

Fix for Issue 90: Enable the WorkQueueItem.timeout to be configurable

Added a new constructor to WorkQueue to enable the WorkQueueItem.timeout to be 
set.

ChangeLog:
----------
* java/com/google/enterprise/connector/common/WorkQueueItem.java:
  - Made timeout protected and non-final so it could be set.

* java/com/google/enterprise/connector/common/WorkQueue.java:
  - Removed a lot of trailing whitespace
  - Created new constructor that would also set the timeout on the WorkQueueItem

* java/com/google/enterprise/connector/scheduler/TraversalScheduler.java:
  - Renamed TraversalWorkQueueItem.timeout instance field to traversalTimeout.
  - Removed dead code related to TraversalWorkQueueItem.timeoutAdditional

* javatests/com/google/enterprise/connector/common/WorkQueueTest.java:
  - Removed trailing whitespace
  - Removed warning by adding a body to the while statement
  - Added tests for work interruption using the new WorkQueue constructor

JavaDoc for new Method:
-----------------------
com.google.enterprise.connector.common.WorkQueue.WorkQueue(int numThreads, long
killThreadTimeout, long workItemTimeout)

Creates a WorkQueue with the given specification and also sets the timeout of
the WorkQueueItem.

Parameters:
    numThreads the number of threads to execute work on the WorkQueue. This
    number should be at least 1.

    killThreadTimeout the additional time in milliseconds given over the
    workItemTimeout before the WorkQueueThread is killed rather than just
    interrupted.

    workItemTimeout time in milliseconds that each WorkQueueItem is given before
    it is interrupted.

How to Use:
-----------
Current default for the killThreadTimeout is 60 seconds and default for the
workItemTimeout is 5 minutes.  If you need to change these then you'll have to
use the new constructor for WorkQueue.

Edit the applicationContext.xml file associated with your Connector Manager
(remember you have to stop Tomcat while you do this and restart it afterward).
Look for the <bean> used to create the WorkQueue.  It should look something
like:

    <bean id="WorkQueue"
        class="com.google.enterprise.connector.common.WorkQueue">
        <constructor-arg index="0" type="int" value="20" />
    </bean>

Change this to use the new constructor.  60 seconds is still a good value for
killThreadTimeout and workItemTimeout to what you think you need.  For example,
a killThreadTimeout of 60 seconds and workItemTimeout of 8 minutes would look
like:

    <bean id="WorkQueue"
        class="com.google.enterprise.connector.common.WorkQueue">
        <constructor-arg index="0" type="int" value="20" />
        <constructor-arg index="1" type="long" value="60 * 1000" />
        <constructor-arg index="2" type="long" value="8 * 60 * 1000" />
    </bean>

Original comment by mgron...@gmail.com on 16 May 2008 at 9:50

GoogleCodeExporter commented 8 years ago

Original comment by mgron...@gmail.com on 19 Jun 2008 at 6:35

GoogleCodeExporter commented 8 years ago 
Note the example above using the math operator inside the long value will not 
work. 
Spring doesn't seem to set the targetType when it creates the TypedStringValue 
during
bean construction and these types of values are are returned as Strings - even 
though
they are specified as type="long".  This results in the following exception 
being thrown:

org.springframework.beans.factory.BeanCreationException: Error creating bean 
with
name 'WorkQueue' defined in ServletContext resource
[/WEB-INF/applicationContext.xml]: 3 constructor arguments specified but no 
matching
constructor found in bean 'WorkQueue' (hint: specify index and/or type 
arguments for
simple parameters to avoid type ambiguities)

The above example should be:

    <bean id="WorkQueue"
        class="com.google.enterprise.connector.common.WorkQueue">
        <constructor-arg index="0" type="int" value="20" />
        <constructor-arg index="1" type="long" value="60000" />
        <constructor-arg index="2" type="long" value="480000" />
    </bean>

Original comment by mgron...@gmail.com on 29 Jan 2009 at 12:35