search

PILLUTLAAVINASH / google-enterprise-connector-manager

Automatically exported from code.google.com/p/google-enterprise-connector-manager
0 stars 0 forks source link

Improved Feed Logging #154

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
Current levels of feed logging are:
 - OFF
 - ALL (verbose: show feed XML - minus content)
 -teedFeedFile (full feed XML - including content - buy stock in Seagate verbosity)

This proposal adds two new levels of feed logging:

INFO - one line for each document processed, including connector name, docid, 
condensed bits of information, and success/failure 
status.

WARNING - one line for each failed document.     This won't address failures on 
the GSA (see Issue 151 ), nor will it track documents 
skipped internally by a connector.  But it will list documents returned by 
DocumentList.nextDocument() that fail for any reason upto an 
unsuccessful return from Pusher.take().

This can leverage the new MDC logging capability.  The MDC map would contain a 
dozen or so bits of information about the current 
document being processed: connector name, document id, add or delete, exception 
thrown, etc.  The map would be cleared before 
each call to nextDocument, and populated as each trackable bit if information 
is acquired.

We could use com.google.enterprise.connector.logging.SimpleFormatter with a 
custom one-line format to produce the output.  To 
maintain brevity, I suggest a flag "word" containing 1-character indicators for 
some of the document's properties.  For instance:

 + : add
 - : delete
 P : is Public
 0 : no content
 U : URL content
 S : Skipped

etc.

Example output might resemble:

2009-05-30 08:36:15  (+0P) [connectorname]  id=0084396236FAB  
application/msword 

or

2009-05-30 08:36:15  (SP) [connectorname]  id=0084396236FAB  FAILED: 
IOException while reading content: skipping...

Original issue reported on code.google.com by Brett.Mi...@gmail.com on 30 May 2009 at 3:50

GoogleCodeExporter commented 8 years ago

Original comment by mgron...@gmail.com on 4 Aug 2009 at 11:56

GoogleCodeExporter commented 8 years ago

Original comment by jl1615@gmail.com on 18 Sep 2009 at 8:35

GoogleCodeExporter commented 8 years ago 
Roll this into Jeff's reporting requirements.

Original comment by jl1615@gmail.com on 19 Sep 2009 at 3:44

GoogleCodeExporter commented 8 years ago 
Should be marked as a duplicate of the Enhancement for reporting requirements 
when that 
issue is entered.

Original comment by mar...@google.com on 4 Dec 2009 at 11:21

GoogleCodeExporter commented 8 years ago 
This issue is filed as Google issue #6513936

Original comment by tdnguyen@google.com on 18 May 2012 at 9:24