search

googlegsa / livelink.v3

Google Search Appliance Connector for Livelink
Apache License 2.0
2 stars 8 forks source link

Sparse traversals may enter an infinite loop #31

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?

1. Traverse a repository with lots of excluded content (e.g., many items in the 
system or 
workflow volumes).

or:

1. Traverse a repository with a traversal user that has very limited access to 
the contents.

What is the expected output? What do you see instead?

The Livelink connector tries to skip over excluded or inaccessible content 
quickly, but if there are 
thousands of such items in a row, by ModifyDate order, it is possible for the 
connector to enter 
an infinite loop. The traversal batch will continue past the timeout until it 
finds content, and the 
connector manager will then drop the found content on the floor and resume the 
traversal from 
the previous checkpoint.

Please use labels and text to provide additional information.

You can detect the infinite loop by looking for log entries containing "RESUME 
TRAVERSAL", 
"SKIPPING PAST", or "RESULTSET". Check for a RESUME TRAVERSAL entry followed by 
a number of 
SKIPPING PAST entries, then a single RESULTSET entry, then another RESUME 
TRAVERSAL entry 
with the same checkpoint string as the previous RESUME TRAVERSAL entry. Here's 
a simplified 
example:

FINE: RESUME TRAVERSAL: 100 rows from 2001-07-27 11:03:10,879496,2008-09-10 15:2
7:54,31773462.
FINER: SKIPPING PAST 2001-07-28 10:14:38,552306,2008-09-10 15:27:54,31773462
FINER: SKIPPING PAST 2001-07-28 20:01:15,564315,2008-09-10 15:27:54,31773462
FINER: SKIPPING PAST 2001-07-29 08:04:15,580327,2008-09-10 15:27:54,31773462
FINER: SKIPPING PAST 2001-07-30 08:31:02,588599,2008-09-10 15:27:54,31773462
FINE: RESULTSET: 71 rows.  DELETESET: 0 rows.
FINE: RESUME TRAVERSAL: 100 rows from 2001-07-27 11:03:10,879496,2008-09-10 15:2
7:54,31773462.

A real example would typically have many more SKIPPING PAST log entries. Two 
specific 
examples from customers had 37 and 54 SKIPPING PAST entries between the 
identical RESUME 
TRAVERSAL entries.

If you encounter this problem, please contact Google Enterprise Support.

Original issue reported on code.google.com by jl1615@gmail.com on 17 Oct 2008 at 11:18

GoogleCodeExporter commented 9 years ago 
Fixed in r284.

Original comment by jl1615@gmail.com on 21 Oct 2008 at 6:29

GoogleCodeExporter commented 9 years ago

Original comment by jl1615@gmail.com on 21 Oct 2008 at 6:30