search

mmx900 / ManalithBot

ManalithBot is an open source IRC bot based on the PircBotX Framework.
http://manalith.org
GNU General Public License v3.0
3 stars 2 forks source link

uriinfo - 연결 실패에 대한 처리 #84

Closed changwoo closed 11 years ago

changwoo commented 11 years ago

종종 연결이 실패하는 경우가 있다. 이 경우 현재는 아무런 출력도 되지 않는데 이 예외를 처리해 다음과 같이 처리 가능.

(1) 일정한 회수만큼 다시 시도하거나 (2) 에러 표시

mmx900 commented 11 years ago

몇 개의 케이스 첨부합니다.

404 status 에러

[18:04:01][TRACE](EventDispatcher.java:103) - MESSAGE : #setzer / setzer / setzer / 10.0.0.1 / http://blog.naver.com/hu
[18:04:01][WARN ](UriInfoPlugin.java:74) - 404 error loading URL http://blog.naver.com/hu
java.io.IOException: 404 error loading URL http://blog.naver.com/hu
        at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:414)
        at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:391)
        at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:157)
        at org.manalith.ircbot.plugin.uriinfo.UriInfoPlugin.getInfo(UriInfoPlugin.java:69)
        at org.manalith.ircbot.plugin.uriinfo.UriInfoPlugin.onMessage(UriInfoPlugin.java:116)
        at org.manalith.ircbot.plugin.EventDispatcher.onMessage(EventDispatcher.java:193)
        at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.pircbotx.hooks.ListenerAdapter.onEvent(ListenerAdapter.java:129)
        at org.pircbotx.hooks.managers.ThreadedListenerManager$1.run(ThreadedListenerManager.java:107)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)

https 에러

[18:07:18][TRACE](EventDispatcher.java:103) - MESSAGE : #setzer / setzer / setzer / 10.0.0.1 / https://kldp.org
[18:07:18][WARN ](UriInfoPlugin.java:74) - sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1868)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1338)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:154)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:804)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:998)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1294)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1321)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1305)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:515)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
        at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:404)
        at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:391)
        at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:157)
        at org.manalith.ircbot.plugin.uriinfo.UriInfoPlugin.getInfo(UriInfoPlugin.java:69)
        at org.manalith.ircbot.plugin.uriinfo.UriInfoPlugin.onMessage(UriInfoPlugin.java:116)
        at org.manalith.ircbot.plugin.EventDispatcher.onMessage(EventDispatcher.java:193)
        at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.pircbotx.hooks.ListenerAdapter.onEvent(ListenerAdapter.java:129)
        at org.pircbotx.hooks.managers.ThreadedListenerManager$1.run(ThreadedListenerManager.java:107)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385)
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
        at sun.security.validator.Validator.validate(Validator.java:260)
        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:126)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1320)
        ... 27 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196)
        at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268)
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380)
        ... 33 more

잘못된 연결

[18:07:07][TRACE](EventDispatcher.java:103) - MESSAGE : #setzer / setzer / setzer / 10.0.1.1 / https://naver.com
[18:07:07][WARN ](UriInfoPlugin.java:74) - 연결이 거부됨
java.net.ConnectException: 연결이 거부됨
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
        at java.net.Socket.connect(Socket.java:579)
        at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:612)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:378)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:473)
        at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:270)
        at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:327)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:931)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
        at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:404)
        at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:391)
        at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:157)
        at org.manalith.ircbot.plugin.uriinfo.UriInfoPlugin.getInfo(UriInfoPlugin.java:69)
        at org.manalith.ircbot.plugin.uriinfo.UriInfoPlugin.onMessage(UriInfoPlugin.java:116)
        at org.manalith.ircbot.plugin.EventDispatcher.onMessage(EventDispatcher.java:193)
        at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at org.pircbotx.hooks.ListenerAdapter.onEvent(ListenerAdapter.java:129)
        at org.pircbotx.hooks.managers.ThreadedListenerManager$1.run(ThreadedListenerManager.java:107)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
mmx900 commented 11 years ago

Jsoup 1.7.1 에서 IOException 대신 HttpStatusException 이나 UnsupportedMimeTypeException 을 사용할 수 있는 것 같습니다. http://jsoup.org/news/release-1.7.1

changwoo commented 11 years ago

https 관련 exception은 사이트의 인증서가 self-signed 인증서이거나 또는 봇이 깔려 있는 컴퓨터에 들어 있는 인증서가 불완전해서인 것으로 보임.

기능으로 볼 때 인증서를 무시해도 별 지장 없어보임. (1) Jsoup에서 인증서를 무시하는 방법을 찾아서 적용하거나 (2) 그런 방법이 없으면 다른 대안을 찾아본다.

changwoo commented 11 years ago

jsoup 교체할 때 방안. #gnome 채널에서.

<fender> changwooo: 묘가 추천하건 httpunit <fender> 단위테스트 하려고 만든 건데 그런 용도로 딱일 것 <fender> 자바스크립트로 타이틀 바꿔도 인식 가능하지 않을까 싶은... ... <changwooo> 이미지 표시는 안 하더라도 네트워크로 읽어오긴 하지 않을까요 <fender> 헤드리스라 최소한 옵션이 있을거고... <changwooo> 그 전에 타이틀만 뽑아낼 수 있으면 되긴 하지만 <fender> 단위테스트할 때 그림까지 바이너리로 비교하진 않을 듯 ... <fender> changwooo: http://httpunit.sourceforge.net/doc/api/com/meterware/httpunit/WebClientListener.html <뒷북요정> [링크 제목] WebClientListener (HttpUnit 1.7 API) <fender> API 훑어봤는데 <fender> 클라이언트 리스너가 있는데... 저걸로 이벤트 붙여서 <fender> responseRecieved에서 WebResponse.getContentType 참조해서 <fender> 아니면 close() <fender> 간단... <fender> getContentType 필요없구나 <fender> WebResponse.isHTML <fender> 무지깔끔... <changwooo> 자자 타이틀 파싱도 검색 <changwooo> myo-based search engine <fender> if (response.isHTML()) response.getTitle()

mmx900 commented 11 years ago

jsoup을 교체하시려는 이유가 contentType을 데이터 다 읽어들인 다음에 판단하는 것이라면요. 1.6.4 이후로 개선이 된 것 같습니다.

개선 당시 : https://github.com/jhy/jsoup/commit/e313f95e19c752a3316048468547af939e19cdd3 현재 코드 : https://github.com/jhy/jsoup/blob/master/src/main/java/org/jsoup/helper/HttpConnection.java#L436

그리고 봇은 최근에 1.6.3에서 1.7.1로 교체 했습니다. https://github.com/mmx900/ManalithBot/commit/6d4138a28ad03477e136f2a39ca3b2d8fe7376e2#ManalithBot/pom.xm

채널에서 몇가지 zip파일 URL로 테스트 해 봤는데 바로 UnsupportedMimeTypeException이 떨어지네요.

crocket commented 11 years ago

그냥 "return e.toString()" 반환하는 것도 괜찮은 것 같은데요?

crocket> http://sdgsdgsdgee 뒷북요정> [링크 제목] 카페24 :: 대한민국 No.1 카페24 호스팅 crocket요정> java.net.UnknownHostException: sdgsdgsdgee

아니면

http://jsoup.org/apidocs/org/jsoup/Connection.html#execute%28%29 여기에서 볼 수 있듯이

MalformedURLException - if the request URL is not a HTTP or HTTPS URL, or is otherwise malformed HttpStatusException - if the response is not OK and HTTP response errors are not ignored UnsupportedMimeTypeException - if the response mime type is not supported and those errors are not ignored SocketTimeoutException - if the connection times out IOException - on error

여기에 대한 Exception handling code를 집어넣어주면 될 것 같네요.

mmx900 commented 11 years ago

HTTPS는 #100 에서 처리하기로 했으므로 이 이슈는 닫습니다.