code4craft / webmagic

A scalable web crawler framework for Java.
http://webmagic.io/
Apache License 2.0
11.45k stars 4.18k forks source link

javax.net.ssl.SSLHandshakeException: Received fatal alert: protocol_version #1097

Closed keatonLiu closed 1 year ago

keatonLiu commented 2 years ago

爬取TLSv1.3的网站时提示Received fatal alert: protocol_version java版本:jdk11 jvm参数:-Djdk.tls.client.protocols=TLSv1,TLSv1.2,TLSv1.1,SSLv3,TLSv1.3 java.security中也解除了jdk.certpath.disabledAlgorithms对TLS1.3的限制

2022-11-24 16:31:40.575 [ pool-17-thread-1 ] - [ WARN ] [ gl.zz.craw.util.CommonDownloader : 91 ] - download page https://nkzykcsz.mh.chaoxing.com/ error javax.net.ssl.SSLHandshakeException: Received fatal alert: protocol_version at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:356) at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293) at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:202) at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1382) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1295) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:416) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:388) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at us.codecraft.webmagic.downloader.HttpClientDownloader.download(HttpClientDownloader.java:85) at gl.zz.craw.util.CommonDownloader.download(CommonDownloader.java:44) at us.codecraft.webmagic.Spider.processRequest(Spider.java:404) at us.codecraft.webmagic.Spider.access$000(Spider.java:61) at us.codecraft.webmagic.Spider$1.run(Spider.java:320) at us.codecraft.webmagic.thread.CountableThreadPool$1.run(CountableThreadPool.java:74) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834)

keatonLiu commented 2 years ago

另一种报错情况

2022-11-24 16:39:39.381 [ pool-21-thread-1 ] - [ WARN ] [ gl.zz.craw.util.CommonDownloader : 91 ] - download page https://www.yiban.cn/school/index/id/5013 error javax.net.ssl.SSLException: Received fatal alert: internal_error at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:133) at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:356) at java.base/sun.security.ssl.Alert$AlertConsumer.consume(Alert.java:293) at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:202) at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1382) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1295) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:416) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:388) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:436) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:384) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:376) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:393) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at us.codecraft.webmagic.downloader.HttpClientDownloader.download(HttpClientDownloader.java:85) at gl.zz.craw.util.CommonDownloader.download(CommonDownloader.java:44) at us.codecraft.webmagic.Spider.processRequest(Spider.java:404) at us.codecraft.webmagic.Spider.access$000(Spider.java:61) at us.codecraft.webmagic.Spider$1.run(Spider.java:320) at us.codecraft.webmagic.thread.CountableThreadPool$1.run(CountableThreadPool.java:74) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834)

sutra commented 2 years ago

webmagic 版本号?更新一下再试试。

keatonLiu commented 1 year ago

用的最新版0.8

sutra commented 1 year ago

应该在 fe3d52e2a439d76025c26d258bd7f74472947548 修好了吧。