reckart
commented
4 years ago @ksuderman - looks like LAPPSGrid is failing again?
Closed UWinch closed 3 years ago
reckart
commented
4 years ago @ksuderman - looks like LAPPSGrid is failing again?
ksuderman
commented
4 years ago I was just about to email you to keep an eye out for problems... I am in the process of moving services off of our failing hardware to a new server. I will check if the service is down or if its a DNS problem...
reckart
commented
3 years ago @ksuderman the problem with the HTTP Status 408 seems to persist. Any news?
ksuderman
commented
3 years ago Sorry. These are the last breaking changes as I rewire DNS and update proxies to point to the new service locations. I hope to have everything back online later today.
ksuderman
commented
3 years ago DNS, proxies, and Apache should all be playing together now. Can you verify things work for you again?
I will be doing recovery testing this week, but all services are running on the new hardware and should be much more reliable.
[Update] I spoke to soon, there is still something amiss.
reckart
commented
3 years ago @ksuderman at the moment our tests seem to be running again. They talk to:
ABNER 1.0.0edu.cmu.lti.oaqa.lapps.LingpipeNER - 1.1.1-SNAPSHOTreckart
commented
3 years ago Since the tests seem to work at the moment and since the tests are now written in such a way that the fail softly if the LAPPS services do not respond (i.e. they do not fail our build if LAPPS is not available), I'll close this issue.
@ksuderman thanks for the response! Let's hope things are running more stable from here on.
ksuderman
commented
3 years ago That is great news!
I was finally able to bend Apache to my will; it turns out it was dropping authentication headers while doing redirects/proxying that took much head scratching to diagnose. Here's to hoping the new hardware is more stable!
reckart
commented
3 years ago @ksuderman Looks like LAPPS is down again:
de.tudarmstadt.ukp.inception.recommendation.imls.lapps.LappsGridRecommenderConformityTest.testNerConformity(LappsGridService{name='ABNER 1.0.0', description='ABNER', url='http://vassar.lappsgrid.org/invoker/anc:gate.abner_1.0.0'}) [0]
de.tudarmstadt.ukp.inception.recommendation.api.recommender.RecommendationException: Cannot predict
at de.tudarmstadt.ukp.inception.recommendation.imls.lapps.LappsGridRecommenderConformityTest.predict(LappsGridRecommenderConformityTest.java:131)
at de.tudarmstadt.ukp.inception.recommendation.imls.lapps.LappsGridRecommenderConformityTest.testNerConformity(LappsGridRecommenderConformityTest.java:96)
Caused by: java.lang.RuntimeException:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Unavailable</title>
</head><body>
<h1>Service Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
<hr>
<address>Apache/2.4.29 (Ubuntu) Server at vassar.lappsgrid.org Port 80</address>
</body></html>Sigh... the services should be back. Now the ServiceManager is crashing so I've added a cron task to restart it nightly. If the SeviceManager continues to be problematic we will take it out of the mix and give you different URLs to access the services.
reckart
commented
3 years ago @ksuderman The same problem again:
de.tudarmstadt.ukp.inception.recommendation.api.recommender.RecommendationException: Cannot predict
at de.tudarmstadt.ukp.inception.recommendation.imls.lapps.LappsGridRecommenderConformityTest.predict(LappsGridRecommenderConformityTest.java:131)
at de.tudarmstadt.ukp.inception.recommendation.imls.lapps.LappsGridRecommenderConformityTest.testNerConformity(LappsGridRecommenderConformityTest.java:96)
Caused by: java.lang.RuntimeException: <!doctype html><html lang="en"><head><title>HTTP Status 404 – Not Found</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 404 – Not Found</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Message</b> The requested resource [/service_manager/invoker/anc:gate.abner_1.0.0] is not available</p><p><b>Description</b> The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.</p><hr class="line" /><h3>Apache Tomcat/9.0.41</h3></body></html>
at de.tudarmstadt.ukp.inception.recommendation.imls.lapps.LappsGridRecommenderConformityTest.predict(LappsGridRecommenderConformityTest.java:131)
at de.tudarmstadt.ukp.inception.recommendation.imls.lapps.LappsGridRecommenderConformityTest.testNerConformity(LappsGridRecommenderConformityTest.java:96)
Describe the bug The unit tests fail due to: "The client did not produce a request within the time that the server was prepared to wait". We had this bug before when the service was not reachable.
HTTP Status 408 -
type Status report
message
description The client did not produce a request within the time that the server was prepared to wait ().
Apache Tomcat/6.0.35
at jp.go.nict.langrid.client.soap.SoapClientFactory$AxisStublessInvocationHandler.invoke(SoapClientFactory.java:106) at com.sun.proxy.$Proxy26.execute(Unknown Source) at org.lappsgrid.client.AbstractClient.execute(AbstractClient.java:74) at de.tudarmstadt.ukp.inception.recommendation.imls.lapps.LappsGridRecommender.predict(LappsGridRecommender.java:80) ... 29 more ```We already test if the service is reachable, check if this method can be modified for this case as well.