search

rajgithub123 / google-enterprise-connector-sharepoint

Automatically exported from code.google.com/p/google-enterprise-connector-sharepoint
0 stars 0 forks source link

The time at which connector completes a crawl cycle should be logged in state file #90

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Have a SharePoint cite collection which has large no. of site 
collections (2000 odd)
2. Deploy the Google Services for SharePoint on SharePoint server
3. Create a connector instance and run it

What is the expected output? 
Looking at the connector logs, user should be able to identify when did the 
connector complete its current crawl cycle

What do you see instead?
Since the rollover nature of log files, the log statement time when 
connector completes its crawl cycle is lost.

The user has no idea as to when did it complete a crawl cycle

The flag in the state file becomes momentarily true and then is set to 
false once the new crawl cycle begins

The connector should log the date and time in the state file when the crawl 
cycle is completed. This is an easy way to identify when did the conenctor 
complete its crawl cycle the last time and gives a good measure as to how 
much time is required to index a complete repository

Original issue reported on code.google.com by rakeshs101981@gmail.com on 11 Aug 2009 at 2:29

GoogleCodeExporter commented 9 years ago

Original comment by rakeshs101981@gmail.com on 11 Aug 2009 at 2:29

GoogleCodeExporter commented 9 years ago

Original comment by rakeshs101981@gmail.com on 11 Aug 2009 at 2:30

GoogleCodeExporter commented 9 years ago 
How about logging the date time and time when the site,list was last traversed 
by the 
connector? 

This will help customers to accurately know when will the connector pick-up 
changes for 
any given site and list. Right now the state file just indicates the last 
crawled site 
and list. 

This is important for customers which have many sites and many lists under a 
given site

Original comment by rakeshs101981@gmail.com on 15 Sep 2009 at 1:05

GoogleCodeExporter commented 9 years ago

Original comment by rakeshs101981@gmail.com on 25 Sep 2009 at 2:11

GoogleCodeExporter commented 9 years ago 
Added timestamp to state file:
1. At global level indicating the last time connector completed its traversal
  <FullRecrawlFlag ID="true" LastFullCrawlDateTime="2009-09-21 15:21:22 PDT"/>

2. At site level indicating when was the site last crawled by connector for 
updates 
or discovering any new child sites
  <WebState ID="http://test.com:12084/sites/testissue85" 
InsertionTime="20090914T175459.627-0700" LastCrawledDateTime="2009-09-30 
17:33:22 
PDT" SPType="sp2007" URL="http://test.com:12084/sites/testissue85" 
WebTitle="testissue85">

3. At list level indicating when was the list last crawled by connector for 
updates

<ListState ChangeToken="1;3;4762db78-de83-4729-ba0d-
fc3b1d066e23;633899536637030000;1507" 
ID="{4762DB78-DE83-4729-BA0D-FC3B1D066E23}" 
LastCrawledDateTime="2009-09-30 17:33:21 PDT" 
LastModified="20090930T172743.000-0700" 
Type="DocumentLibrary" 
URL="http://test.com:12084/sites/testissue85/testdoclib/Forms/AllItems.aspx">
<LastDocCrawled ID="{4762DB78-DE83-4729-BA0D-FC3B1D066E23}" 
LastModified="20090930T172743.000-0700"/>
</ListState>

More details can be found here:

http://code.google.com/p/google-enterprise-connector-sharepoint/source/detail?r=
323
http://code.google.com/p/google-enterprise-connector-sharepoint/source/detail?r=
335
http://code.google.com/p/google-enterprise-connector-sharepoint/source/detail?r=
364

Original comment by rakeshs101981@gmail.com on 1 Oct 2009 at 10:28

GoogleCodeExporter commented 9 years ago

Original comment by rakeshs101981@gmail.com on 5 Nov 2009 at 9:00

GoogleCodeExporter commented 9 years ago 
Verified in 2.4 Release

Original comment by ashwinip...@gmail.com on 14 Dec 2009 at 6:22