Deleting a connector may leave behind connector state.

[GoogleCodeExporter]

What steps will reproduce the problem?
1. In the GSA admin console, create a connector instance.
2. While the connector is in the middle of processing a batch, delete the
connector through the admin console.

What is the expected output? What do you see instead?

The expected output is for the connector configuration directory, the
connector schedule, and the connector state to be removed. The first two
are removed, but the connector state is not. Actually, it *is* removed, and
then stored again.

What version of the product are you using? On what operating system?

Hosted GSA at 8.6.49.37, Connector Manager R449.

Please provide any additional information below.

The problem is due to a change in revision 334 of the Connector Manager
that affects the removal of connectors. In
ProductionManager.removeConnector, the schedule and state are removed, and
then the connector is removed from the scheduler. The scheduler interrupts
the work queue thread, if one is active. But if the work queue thread is
executing the while loop in runBatch, the interruption will trigger a call
to checkpointAndSave, which will write the new connector state to the
preferences, and it won't be deleted.

If another connector of the same name is created, it will use the undeleted
connector state rather than starting a traversal from scratch. The stored
connector state is not tied to the connector configuration, or even the
connector type, so the pre-existing checkpoint may be invalid for the new
connector, or simply interfere with the correct traversal of the repository.

Original issue reported on code.google.com by jl1615@gmail.com on 27 May 2007 at 12:19

[GoogleCodeExporter]

Original comment by donald.z...@gmail.com on 18 Apr 2008 at 10:15

• Changed state: Accepted
• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

This is also Google bug 752976.

The workaround for this bug, but not for the related issue 87, is to pause the 
traversal before deleting the connector 
instance: 

1. Edit the connector instance.
2. Set the schedule from 1:00 AM to 1:00 AM (with only one schedule line).
3. Wait a few minutes.
4. Delete the connector instance.

Original comment by jl1615@gmail.com on 7 May 2008 at 3:03

[GoogleCodeExporter]

Fixed in revision r809 .

When a connector was deleted, its persistent ConnectorStateStore
(the stored Checkpoint) and ConnectorScheduleStore are deleted.
However, a slow-to-die worker thread may, as its last gasp,
rewrite the ConnectorStateStore.  As a result, deleting and
recreating a connector would often result in the new connector
starting up where the old connector left off, rather than
starting from the beginning.

Many layers of abstraction make it difficult to communicate
deleted connector status down to the worker threads, so we
decided to provide a mechanism that can enable and disable
the ConnectorStateStore for a specific connector.  The CM
then disables the store when a connector is deleted, and
enables the store when a connector is created.  Attempting
to read or write a disabled store throws an exception.

Similar changes were not needed for the ConnectorScheduleStore
because it is not managed asynchronously (yet?).

ChangeLog:
---------

* java/com/google/enterprise/connector/persist/ConnectorStateStore.java:
  - Add enable and disable methods to the ConnectorStateStore Interface.

* java/com/google/enterprise/connector/persist/PrefsStore.java:
  - Implement enable and disable methods in the ConnectorStateStore
    Implementation.

* java/com/google/enterprise/connector/instantiator/SpringInstantiator.java:
  - Enable a connector's ConnectorStateStore when creating a new
    instance of a connector.  See also the version of SpringInstantiator
    that fixes Issues 35, 60, and 63.  When both branches are merged
    into the trunk, that version of SpringInstantiator should be used
    (with the commented out Issue 47 line restored);  not this one.
  - Disable a connector's ConnectorStateStore when deleting the connector.

* java/com/google/enterprise/connector/manager/ProductionManager.java:
  - Remove the connector from the scheduler before deleting the scheduleStore.

* java/com/google/enterprise/connector/traversal/QueryTraverser.java:
  - Catch IllegalStateExceptions when reading or writing a connector's
    state.  These exceptions mean a connector has ceased to exist and
    they are dealt with accordingly.

* javatests/com/google/enterprise/connector/instantiator/MockInstantiator.java:
  - Enable a connector's ConnectorStateStore when creating a new
    connector and disable it when deleting a connector.
  - Add dropConnector() implementation, replacing stub.

* 
javatests/com/google/enterprise/connector/persist/MockConnectorStateStore.java:
  - Implement enable and disable methods in the mock ConnectorStateStore
    implementation.

* javatests/com/google/enterprise/connector/persist/PrefsStoreTest.java:
  - Add tests for the enable/disable functionality.

Original comment by Brett.Mi...@gmail.com on 28 May 2008 at 10:31

• Changed state: Fixed

[GoogleCodeExporter]

Original comment by mgron...@gmail.com on 19 Jun 2008 at 6:35

• Added labels: Milestone-Release_1.1.0