Can't recrawl an existing connector

[GoogleCodeExporter]

What steps will reproduce the problem?

See below.

What is the expected output? What do you see instead?

It would be extremely helpful for development, and nice for production if a
connector could be recrawled from scratch.

Please use labels and text to provide additional information.

Original issue reported on code.google.com by donald.z...@gmail.com on 24 Jan 2007 at 3:48

[GoogleCodeExporter]

Google Bug #244000

Original comment by vjo...@gmail.com on 9 Feb 2007 at 5:15

[GoogleCodeExporter]

You can recrawl a connector instance by deleting the connector instance and
recreating it. While it would certainly be less tedious to have a link to click
instead, this isn't a bad workaround.

Original comment by jl1615@gmail.com on 17 Feb 2007 at 12:12

[GoogleCodeExporter]

Original comment by donald.z...@gmail.com on 18 Apr 2008 at 10:07

• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

It should be easy to force a restart, but it is not quite so.

Simply blowing away the connector state store for a connector should force a 
call to
startTraversal.  However, Issue 47 demonstrated that straggling worker threads 
may
rewrite the connector state after it has been deleted, and Issue 94 shows that
connectors that store their state internally would not notice the removal of the
connector state.

Original comment by Brett.Mi...@gmail.com on 6 Jun 2008 at 12:43

[GoogleCodeExporter]

Fixed in revision r879

This set of changes addresses Connector Manager Issue 25 - 
Forcing a Connector to re-traverse the document repository.
This is a useful tool when troubleshooting customer-specific
configuration issues.  It is also useful when the customer
changes certain configuration options, such as adding or 
deleting folders to index or changing the traversal user.

Much of this change supercedes some of the changes made
to fix Issue 47.  This implementation not only fixes 
Issue 47, but also addresses this problem and a potential
fix for hanging worker threads.

This feature is provided by a Connector Manager Servlet
command, restartConnectorTraversal, supplying it the name 
of the connector instance to restart.

Usage:
-----
To restart the indexing traversal for a Connector:
  http://[cm_host_addr]/connector-manager/restartConnectorTraversal?ConnectorName=[connector_name]
where [connector_name] is the name of a connector instance.

Change Log:
----------

M webdocs/web.xml
  - Add description for RestartConnectorTraversal servlet.

M java/com/google/enterprise/connector/instantiator/Instantiator.java
  - Prototype restartConnectorTraversal() method interface.

M java/com/google/enterprise/connector/instantiator/SpringInstantiator.java
  - Replace ConnectorStateStore.enable() and .disable() calls with
    GenerationalStateStore calls.

  - Implement restartConnectorTraversal() method.  Deletes the ConnectorStateStore
    for the connector and bumps the generation, preventing any outstanding
    workers from re-writing the store.  The null ConnectorStateStore will
    force the connector's startTraversal() method to be called.

A java/com/google/enterprise/connector/servlet/RestartConnectorTraversal.java
  - New servlet interface provides a mechanism to restart the traversal.

A java/com/google/enterprise/connector/persist/GenerationalStateStore.java
  - Wrapper over a ConnectorStateStore that adds the concept of 'generations'
    to the store.  Once a new generation is created, older generations are
    prevented from reading or writing the store.

M java/com/google/enterprise/connector/persist/ConnectorStateStore.java
  - Remove the enable() and disable() methods from the interface.

M java/com/google/enterprise/connector/persist/PrefsStore.java
  - Remove the enable() and disable() methods from the implementation.

M java/com/google/enterprise/connector/manager/Manager.java
  - Add restartConnectorTraversal() method prototype.

M java/com/google/enterprise/connector/manager/ProductionManager.java
  - Add restartConnectorTraversal() method implementation.  It is an
    intermediate between the Servlet and the Instantiator.

M javatests/com/google/enterprise/connector/instantiator/MockInstantiator.java
  - Replace ConnectorStateStore.enable() and .disable() calls with
    GenerationalStateStore calls.

  - Implement restartConnectorTraversal() method. Deletes the ConnectorStateStore
    for the connector and bumps the generation, preventing any outstanding
    workers from re-writing the store.  The null ConnectorStateStore will
    force the connector's startTraversal() method to be called.

A 
javatests/com/google/enterprise/connector/persist/GenerationalStateStoreTest.jav
a
  - Test the Generational Connector State Store implementation.

M javatests/com/google/enterprise/connector/persist/MockConnectorStateStore.java
  - Remove the enable() and disable() methods from the implementation.

M javatests/com/google/enterprise/connector/persist/PrefsStoreTest.java
  - Remove the testEnableDisable() test.

M javatests/com/google/enterprise/connector/manager/MockManager.java
  - Add a no-op restartConnectorTraversal() method implementation.

Original comment by Brett.Mi...@gmail.com on 3 Jul 2008 at 8:07

• Changed state: Fixed

[GoogleCodeExporter]

Original comment by mgron...@gmail.com on 3 Jul 2008 at 4:57

• Added labels: Milestone-Release_1.1.0