search

PILLUTLAAVINASH / google-enterprise-connector-manager

Automatically exported from code.google.com/p/google-enterprise-connector-manager
0 stars 0 forks source link

New backlogcount check throws an IOException upon startup when the feed host is not defined #177

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
What steps will reproduce the problem?
1. Install a new WAR file built from head (Revision: 2197)
2. Start up the hosting Tomcat Server
3. Take a look in the logs

What is the expected output?

Logs should show clean startup.

What do you see instead?

SEVERE IOException related to checking the backlogcount is logged upon
startup.  This looks bad and might prompt some support calls:

Aug 10, 2009 4:18:46 PM com.google.enterprise.connector.servlet.StartUp init
INFO: init done
Aug 10, 2009 4:18:46 PM
com.google.enterprise.connector.pusher.GsaFeedConnection getBacklogCount
FINEST: Opening backlogcount connection.
Aug 10, 2009 4:18:46 PM
com.google.enterprise.connector.pusher.GsaFeedConnection getBacklogCount
SEVERE: IOException while reading backlogcount
java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
    at java.net.Socket.connect(Socket.java:507)
    at java.net.Socket.connect(Socket.java:457)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:365)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:477)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:214)
    at sun.net.www.http.HttpClient.New(HttpClient.java:287)
    at sun.net.www.http.HttpClient.New(HttpClient.java:299)
    at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.j
ava:792)
    at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:
744)
    at
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:669)
    at
com.google.enterprise.connector.pusher.GsaFeedConnection.getBacklogCount(GsaFeed
Connection.java:341)
    at
com.google.enterprise.connector.pusher.GsaFeedConnection.isBacklogged(GsaFeedCon
nection.java:297)
    at
com.google.enterprise.connector.scheduler.HostLoadManager.shouldDelay(HostLoadMa
nager.java:226)
    at
com.google.enterprise.connector.scheduler.TraversalScheduler.run(TraversalSchedu
ler.java:166)
    at java.lang.Thread.run(Thread.java:595)
Aug 10, 2009 4:18:46 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-7077

Is it possible to detect the feed host is not set to prevent throwing this
type of exception?

Original issue reported on code.google.com by mgron...@gmail.com on 10 Aug 2009 at 11:27

GoogleCodeExporter commented 8 years ago 
Also right after CM is registered with GSA the following exception is thrown if 
the
GSA is not configured to trust feeds from the CM:

Aug 10, 2009 4:38:20 PM 
com.google.enterprise.connector.servlet.TestConnectivity doGet
INFO: Hello from the TestConnectivity servlet!
Aug 10, 2009 4:38:20 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Opening backlogcount connection.
Aug 10, 2009 4:38:20 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Received backlogcount: Error - Unauthorized Request
Aug 10, 2009 4:38:20 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
SEVERE: Invalid backlogcount: Error - Unauthorized Request
java.lang.NumberFormatException: For input string: "Error - Unauthorized 
Request"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
        at java.lang.Integer.parseInt(Integer.java:447)
        at java.lang.Integer.parseInt(Integer.java:497)
        at
com.google.enterprise.connector.pusher.GsaFeedConnection.getBacklogCount(GsaFeed
Connection.java:353)
        at
com.google.enterprise.connector.pusher.GsaFeedConnection.isBacklogged(GsaFeedCon
nection.java:297)
        at
com.google.enterprise.connector.scheduler.HostLoadManager.shouldDelay(HostLoadMa
nager.java:226)
        at
com.google.enterprise.connector.scheduler.TraversalScheduler.run(TraversalSchedu
ler.java:166)
        at java.lang.Thread.run(Thread.java:595)

Again, this looks bad in the logs.  Admittedly, it's a potential problem since 
the
GSA will not accept feeds from the CM as configured.  It would be better, 
however, to
log a message to that fact rather than dump the NumberFormatException.

Original comment by mgron...@gmail.com on 10 Aug 2009 at 11:42

GoogleCodeExporter commented 8 years ago 
Actually, anytime the backlog count is checked if the GSA is not configured to 
trust
the CM there will be an exception since it returns 200 and the code is assuming 
an
integer:

# With GSA Feed IP White list limited to localhost
$ curl -i http://xxx.xxx.xxx.149:xxxx/getbacklogcount
HTTP/1.1 200 OK
Content-Type: text/plain
Date: Mon, 10 Aug 2009 23:56:28 GMT
Server: feedergate_1.0
Content-Length: 28

Error - Unauthorized Request

# With CM added to IP White list:
$ curl -i http://xxx.xxx.xxx.149:xxxx/getbacklogcount
HTTP/1.1 200 OK
Content-Type: text/plain
Date: Tue, 11 Aug 2009 00:00:56 GMT
Server: feedergate_1.0
Content-Length: 1

0

Need to remove that assumption from the code.

Original comment by mgron...@gmail.com on 11 Aug 2009 at 12:02

GoogleCodeExporter commented 8 years ago 
Fixed 11 Aug 2009 in Connector Manager revision r2202

Fix NumberFormatException in GsaFeedConnection.getBacklogCount().

The logic was already doing the right thing.  However,
I no longer display error returns from getbacklogcount
servlet as errors w/stackdump in the log.  The returned
message is still logged, although at FINEST w/o a stackdump.

Original comment by Brett.Mi...@gmail.com on 11 Aug 2009 at 5:26

GoogleCodeExporter commented 8 years ago 
Log still fills up with ConnectExceptions if the GSA is not accepting 
connections
from the CM and it tries to check the backlogcount:

Aug 11, 2009 2:36:24 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Opening backlogcount connection.
Aug 11, 2009 2:36:24 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
SEVERE: IOException while reading backlogcount
java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
        at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
        at java.net.Socket.connect(Socket.java:507)
        at java.net.Socket.connect(Socket.java:457)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:365)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:477)
        at sun.net.www.http.HttpClient.<init>(HttpClient.java:214)
        at sun.net.www.http.HttpClient.New(HttpClient.java:287)
        at sun.net.www.http.HttpClient.New(HttpClient.java:299)
        at
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.j
ava:792)
        at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:
744)
        at
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:669)
        at
com.google.enterprise.connector.pusher.GsaFeedConnection.getBacklogCount(GsaFeed
Connection.java:341)
        at
com.google.enterprise.connector.pusher.GsaFeedConnection.isBacklogged(GsaFeedCon
nection.java:297)
        at
com.google.enterprise.connector.scheduler.HostLoadManager.shouldDelay(HostLoadMa
nager.java:226)
        at
com.google.enterprise.connector.scheduler.TraversalScheduler.run(TraversalSchedu
ler.java:166)
        at java.lang.Thread.run(Thread.java:595)

Original comment by mgron...@gmail.com on 11 Aug 2009 at 9:37

GoogleCodeExporter commented 8 years ago 
Fixed 11 Aug 2009 in Connector Manager revision r2206

I removed all Exception logging in GsaFeedConnection getBacklogCount() and 
isFeedAvailable().
Errors are still logged at FINEST level, but only the single line message, not 
a stack backtrace.

Original comment by Brett.Mi...@gmail.com on 12 Aug 2009 at 12:06

GoogleCodeExporter commented 8 years ago 
Tested with GSA using HEAD build.  Log is much more condensed now:

Aug 11, 2009 4:25:23 PM com.google.enterprise.connector.servlet.StartUp init
INFO: init done
Aug 11, 2009 4:25:23 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-7076
Aug 11, 2009 4:40:23 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Opening backlogcount connection.
Aug 11, 2009 4:40:23 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Error while reading backlogcount: Connection refused
Aug 11, 2009 4:55:23 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Opening backlogcount connection.
Aug 11, 2009 4:55:23 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Error while reading backlogcount: Connection refused
Aug 11, 2009 5:10:24 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Opening backlogcount connection.
Aug 11, 2009 5:10:24 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Error while reading backlogcount: Connection refused
Aug 11, 2009 5:25:24 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Opening backlogcount connection.
Aug 11, 2009 5:25:24 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Error while reading backlogcount: Connection refused
Aug 11, 2009 5:40:24 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Opening backlogcount connection.
Aug 11, 2009 5:40:24 PM com.google.enterprise.connector.pusher.GsaFeedConnection
getBacklogCount
FINEST: Error while reading backlogcount: Connection refused

Original comment by mgron...@gmail.com on 12 Aug 2009 at 12:50

GoogleCodeExporter commented 8 years ago

Original comment by jla...@google.com on 9 Dec 2010 at 10:49