All the extra IDs (URLs in case of attachments) which are used during the consrtruction of delete feeds are updated before checkpoint

[GoogleCodeExporter]

As per the fix for Issue 106, any updates in the connector state should be
done at the time of checkpoint. As a follow up to the same, all the
extraIDs whcih are saved into the connector state as "FolderIDs",
"Attachment" etc are updated at the time of checkpoint. Currently, these
get updated at the time of discovery. This leads to sending of unnecessary
delete feeds if all the documents/attachments corresponding to these
ExtraIDs are sent to the CM.

Original issue reported on code.google.com by th.nitendra on 20 Nov 2009 at 6:04

[GoogleCodeExporter]

Original comment by rakeshs101981@gmail.com on 20 Nov 2009 at 2:10

• Added labels: Milestone-2.4

[GoogleCodeExporter]

updating of extraIDs has been moved to checkpoint.
http://code.google.com/p/google-enterprise-connector-sharepoint/source/detail?r=
450
http://code.google.com/p/google-enterprise-connector-sharepoint/source/detail?r=
449

Original comment by th.nitendra on 25 Nov 2009 at 5:23

• Changed state: Fixed

[GoogleCodeExporter]

Fix details:

# Feature or Bug Fix
1.  All the extraIDs for list/list-items/attachments that are saved in the 
physical
state file should be updated at the time of checkpoint.
2.  Attachments are handled differently from their parent list item.
3.  Updated some comments and Javadoc as they were invalid and outdated.

# Problem
1.  ExtraIDs, which are mainly used for sending delete feeds were getting 
updated at
the time of discovery. So, if the list is deleted before all the docs with 
extraIDs
are sent, unnecessary delete feeds will be sent for them.
2.  All the attachments and the their parent list item share the same ID. This 
ID was
getting updated into the deleteCache as soon as the first document (attachment 
or
item) is sent. This stops rest of the documents (attachments/items) from being 
sent
as delete feed. This could be reproduced only when the connector was restarted 
in
between all such docs are sent to CM.

# High level list of changes
Call to updateExtraIDs() and RemoveExtraIDs are shifted during checkpoint call.
Folders are an exception of this because they are not sent as document and hence
cannot be tracked during checkpoint. Hence, updating extraIDs for folders are 
still
done during the document discovery which is well expected.

# Specific flow/logic that the reviewer needs to focus on
/trunk/projects/sharepoint/source/java/com/google/enterprise/connector/sharepoin
t/spiimpl/SPDocument.java
1.  Updated CompareTo() so that attachments are always sent before their parent 
list
item. 
2.  Added a member variable used during checkpoint for updating extraIDs.
/trunk/projects/sharepoint/source/java/com/google/enterprise/connector/sharepoin
t/spiimpl/SPDocumentList.java
1.  Moved the whole update/remove extraIDs logic during checkpoint.
/trunk/projects/sharepoint/source/java/com/google/enterprise/connector/sharepoin
t/state/ListState.java
1.  Updated Javadoc
/trunk/projects/sharepoint/source/java/com/google/enterprise/connector/sharepoin
t/state/WebState.java

1.  Sorting of deleted documents so that item is sent after the attachments
/trunk/projects/sharepoint/source/java/com/google/enterprise/connector/sharepoin
t/wsclient/ListsWS.java

1.  Removed all the updating/removing of extraIDs logic
2.  Updated one of the log message as it was wrong as per the implementation.
/trunk/projects/sharepoint/source/javatests/com/google/enterprise/connector/shar
epoint/state/ListStateTest.java
1.  Updated as per the change in the ListState.java

# Testing: How has the change being tested
Manually by running the connector and checking the way extraIDs are
added/removed/updated. Also ensured that delete feeds for all the attachments 
and
listitem is sent even if the connector is restarted.

# Revision
http://code.google.com/p/google-enterprise-connector-sharepoint/source/detail?r=
449

Original comment by th.nitendra on 25 Nov 2009 at 5:36

[GoogleCodeExporter]

Verified in 2.4 Release

Original comment by ashwinip...@gmail.com on 14 Dec 2009 at 6:43

• Changed state: Verified