search

opengeospatial / teamengine

TEAM Engine (Test, Evaluation, And Measurement Engine) is an engine for testing web services and other resources written in JAVA. It executes test scripts written in Compliance Test Language (CTL), TestNG and other languages. It is lightweight and easy to run as a command line or to setup as a service. It can be used to test any type of service or encoding. It is also the official tool used by the Open Geospatial Consortium (OGC) for compliance testing.
Apache License 2.0
45 stars 41 forks source link

Enhance TEAM Engine to be capable of validating resources retrieved via HTTPS protocol #255

Closed dstenger closed 6 years ago

dstenger commented 7 years ago

Already reported for WMS 1.3 test suite: https://github.com/opengeospatial/ets-wms13/issues/27

dstenger commented 7 years ago

We will improve the description/specification of this issue soon.

dstenger commented 6 years ago

Tasks:

  1. Find service which can be used for testing (also see opengeospatial/ets-wms13#27).
  2. Document behaviour without fix (also see opengeospatial/ets-wms13#27).
  3. Enhance Java HTTP client to be capable of requesting resources via HTTPS protocol.
dstenger commented 6 years ago

Validation with following service already works (it is a reference implementation): https://services.interactive-instruments.de/ogc-reference/simple/wfs?request=getcapabilities&service=wfsservices.interactive-instruments.de/ogc-reference/simple/wfs?request=getcapabilities&service=wfs

The ets-wfs20 is TestNG based.

dstenger commented 6 years ago

Also, a CTL based test suite can validate an HTTPS secured service successfully (reference implementation): https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wms130?service=WMS&request=GetCapabilities

The ets-wms13 is CTL based.

dstenger commented 6 years ago

New tasks:

  1. Go through all reference implementations [1], check if they can be accessed via HTTPS, test them and document any occurring errors.
  2. Analyse all occurring errors.
  3. Create fix for identified errors. Also, opengeospatial/ets-wms13#27 should be taken in consideration.

[1] https://github.com/opengeospatial/cite/wiki/Reference-Implementations

keshavnangare commented 6 years ago

Following are the behaviour of the Reference-Implementations:

Standard Organization HTTP End Point HTTPS Status
CSW 2.0.2 Florence Research Association GI-cat 12.0 Not able to access as HTTPS ----
CSW 2.0.2 OSGeo pycsw 2.0 Not able to access as HTTPS ----
CSW 2.0.2 ESRI Geoportal server https://gptogc.esri.com/geoportal/csw?request=GetCapabilities&service=CSW&AcceptVersions=2.0.2 Working
CAT 3.0 OSGeo pycsw 2.0 Not able to access as HTTPS ----
CAT 3.0 ESRI Geoportal server https://gptogc.esri.com/geoportal/csw3?service=CSW&version=3.0.0&request=GetCapabilities Working
GeoPackage 1.0 Luciad LuciadFusion 2015 and LuciadLightspeed 2015 http://www.geopackage.org/data/simple_sewer_features.gpkg Working
GML 3.2.1 lat/lon deegree 3.4 https://services.intera Working
GML 3.2.1 interactive instruments GmbH XtraServer 4.11 https://services.interactive-instruments.de/ogc-reference/schema/XtraServerGetFeature.xml Working
SOS 2.0 52North 52North https://sensorweb.demo.52north.org/52n-sos-webapp/service?service=SOS&request=GetCapabilities&AcceptVersions=2.0.0 Failed with ssl
WCS 2.0.1 rasdaman rasdaman Not able to access as HTTPS ----
WCS 2.0.1 OSGeo MapServer http://ows.eox.at/cite/mapserver?service=wcs&version=2.0.1&request=getcapabilities Working
EO-WCS 1.0 EOX IT Services GmbH EOxServer https://ows.eox.at/cite/eoxserver/ows?service=wcs&version=2.0.1&request=getcapabilities Working
WMS 1.1.1 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wms111?service=WMS&request=GetCapabilities Failed with SSLHandShake
WMS 1.1.1 OSGeo GeoServer Do not support HTTPS ----
WMS 1.1.1 interactive instruments GmbH XtraServer 4.11 https://services.interactive-instruments.de/ogc-ref Working
WMS 1.3.0 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wms130?service=WMS&request=GetCapabilities Working
WMS 1.3.0 OSGeo GeoServer Do not support HTTPS ----
WMS 1.3.0 interactive instruments GmbH XtraServer 4.11 https://services.interactive-instruments.de/ogc-reference/cite/wms?request=getcapabilities&service=wms&version=1.3 Working
WMTS 1.0.0 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wmts100?service=WMTS&request=GetCapabilities Failed with SSLHandShake in local Environment but worked in cite-beta environment
WFS 1.0.0 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wfs100?service=WFS&request=GetCapabilities Failed with SSLHandShake in local Environment but worked in cite-beta environment
WFS 1.1.0 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wfs110?service=WFS&request=GetCapabilities Failed with SSLHandShake in local Environment but worked in cite-beta environment
WFS 1.1.0 interactive instruments GmbH XtraServer 4.11 https://services.interactive-instruments.de/ogc-reference/simple110/wfs?request=getcapabilities&service=wfs Working
WFS 2.0.0 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wfs200?service=WFS&request=GetCapabilities Working
WFS 2.0.0 interactive instruments GmbH XtraServer 4.11 https://services.interactive-instruments.de/ogc-reference/simple/wfs?request=getcapabilities&service=wfs Working
WFS 2.0.0 Avitech GmbH Avitech SWIM (AxL) 4.0 - WFS https://wfst.axl.aero/AxlRest/wfs?service=WFS&version=2.0.0&request=GetCapabilities Failed with SSLHandShake
dstenger commented 6 years ago

All test services were taken from: https://github.com/opengeospatial/cite/wiki/Unofficial-OGC-Reference-Implementations

dstenger commented 6 years ago

Analyse results of table in https://github.com/opengeospatial/teamengine/issues/255#issuecomment-344920430

keshavnangare commented 6 years ago

I have tested on the Ubuntu Linux machine.

Once again I will check WMS 1.1.1 whether it is reproducing or not.

keshavnangare commented 6 years ago

@dstenger

I have tested on the Windows 8.1 environment stilI getting the same error.

Test: WMS 1.1.1
[Service-URL] https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wms111?service=WMS&request=GetCapabilities
Test Name : wms:wms_main type Mandatory
net.sf.saxon.s9api.SaxonApiException: Error in call to extension function {public org.w3c.dom.NodeList com.occamlab.te.TECore.request(org.w3c.dom.Document,java.lang.String) throws java.lang.Throwable}: Exception in extension function javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at net.sf.saxon.s9api.XsltTransformer.transform(XsltTransformer.java:192)
    at com.occamlab.te.TECore.executeTemplate(TECore.java:607)
    at com.occamlab.te.TECore.executeTest(TECore.java:818)
    at com.occamlab.te.TECore.execute_test(TECore.java:426)
    at com.occamlab.te.TECore.execute_suite(TECore.java:474)
    at com.occamlab.te.TECore.execute(TECore.java:303)
    at com.occamlab.te.TECore.run(TECore.java:2406)
    at java.lang.Thread.run(Thread.java:745)
Caused by: net.sf.saxon.trans.XPathException: Error in call to extension function {public org.w3c.dom.NodeList com.occamlab.te.TECore.request(org.w3c.dom.Document,java.lang.String) throws java.lang.Throwable}: Exception in extension function javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at net.sf.saxon.functions.ExtensionFunctionCall.iterate(ExtensionFunctionCall.java:151)
    at net.sf.saxon.expr.PathExpression.iterate(PathExpression.java:848)
    at net.sf.saxon.sort.DocumentSorter.iterate(DocumentSorter.java:84)
    at net.sf.saxon.instruct.CopyOf.processLeavingTail(CopyOf.java:292)
    at net.sf.saxon.instruct.Instruction.process(Instruction.java:94)
    at net.sf.saxon.instruct.DocumentInstr.evaluateItem(DocumentInstr.java:282)
    at net.sf.saxon.expr.ExpressionTool.evaluate(ExpressionTool.java:295)
    at net.sf.saxon.expr.LetExpression.eval(LetExpression.java:341)
    at net.sf.saxon.expr.LetExpression.process(LetExpression.java:372)
    at net.sf.saxon.instruct.ForEach.processLeavingTail(ForEach.java:300)
    at net.sf.saxon.expr.LetExpression.processLeavingTail(LetExpression.java:551)
    at net.sf.saxon.instruct.Template.applyLeavingTail(Template.java:175)
    at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:343)
    at net.sf.saxon.Controller.transformDocument(Controller.java:1736)
    at net.sf.saxon.Controller.transform(Controller.java:1560)
    at net.sf.saxon.s9api.XsltTransformer.transform(XsltTransformer.java:190)
    ... 7 more
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1509)
    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
    at sun.security.ssl.Handshaker.process_record(Handshaker.java:914)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153)
    at com.occamlab.te.TECore.parse(TECore.java:2089)
    at com.occamlab.te.TECore.parse(TECore.java:2066)
    at com.occamlab.te.TECore.request(TECore.java:1658)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at net.sf.saxon.functions.ExtensionFunctionCall.invokeMethod(ExtensionFunctionCall.java:533)
    at net.sf.saxon.functions.ExtensionFunctionCall.call(ExtensionFunctionCall.java:256)
    at net.sf.saxon.functions.ExtensionFunctionCall.iterate(ExtensionFunctionCall.java:147)
    ... 22 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
    at sun.security.validator.Validator.validate(Validator.java:260)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1491)
    ... 42 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382)
    ... 48 more
dstenger commented 6 years ago

@keshav-nangare Can you please document the used Java version?

keshavnangare commented 6 years ago

Details:

JAVA [Windows 8.1]: 

java version "1.8.0_91"
Java(TM) SE Runtime Environment (build 1.8.0_91-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode)

Tomcat 7

lgoltz commented 6 years ago

I've tested WMS 1.1.1 (https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wms111?service=WMS&request=GetCapabilities) with local docker environment (ETS WMS 1.1 v1.16 and TEAM Engine 5.1) as well as with production. Tests are successful.

@keshav-nangare Can you try again with production?

dstenger commented 6 years ago

@keshav-nangare What implementation of Java are you using? It seems that the trustStore of your Java installation is broken. Can you try to setup a newly installed Java?

keshavnangare commented 6 years ago

@lgoltz

The test wms1.1 is working fine with the production.

@dstenger

OS: Windows 8.1 I'm using Oracle JAVA:

java version "1.8.0_162"
Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)

After updating to above latest version, still getting this error on my local machine.

dstenger commented 6 years ago

@keshav-nangare

Regarding your local system: In following file you can find all trusted certificates of you Java installation: $JAVA_HOME/jre/lib/security/cacerts On Linux following command can be executed to view all entries (please check for the corresponding command on Windows): keytool -list -keystore $JAVA_HOME/jre/lib/security/cacerts How many entries a present in the trustStore of your local Java installation?

Regarding table of comment https://github.com/opengeospatial/teamengine/issues/255#issuecomment-344920430: Can you please repeat all tests on production and just document errors occurring there? Please use a table for documentation again.

keshavnangare commented 6 years ago

@dstenger

I checked the trusted certificates as you suggested and following are the entries:

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 104 entries

Sure, I will repeat all tests on production.

lgoltz commented 6 years ago

The failure described in https://github.com/opengeospatial/teamengine/issues/255#issuecomment-350680643 is also reported in #312

keshavnangare commented 6 years ago

Following are the behaviour of the Reference-Implementations on Production environment:

Standard Organization HTTP End Point HTTPS Status
CSW 2.0.2 Florence Research Association GI-cat 12.0 Not able to access as HTTPS ----
CSW 2.0.2 OSGeo pycsw 2.0 Not able to access as HTTPS ----
CSW 2.0.2 ESRI Geoportal server https://gptogc.esri.com/geoportal/csw?request=GetCapabilities&service=CSW&AcceptVersions=2.0.2 Working
CAT 3.0 OSGeo pycsw 2.0 Not able to access as HTTPS ----
CAT 3.0 ESRI Geoportal server https://gptogc.esri.com/geoportal/csw3?service=CSW&version=3.0.0&request=GetCapabilities Working
GeoPackage 1.0 Luciad LuciadFusion 2015 and LuciadLightspeed 2015 https://demo.luciad.com/GeoPackageData/bluemarble.gpkg Working
GML 3.2.1 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/gml321?service=WFS&request=GetFeature&Version=2.0.0&typenames=app:Autos Working
GML 3.2.1 interactive instruments GmbH XtraServer 4.11 https://services.interactive-instruments.de/ogc-reference/schema/XtraServerGetFeature.xml Working
SOS 2.0 52North 52North https://sensorweb.demo.52north.org/52n-sos-webapp/service?service=SOS&request=GetCapabilities&AcceptVersions=2.0.0 Failed with ssl
SPS 2.0 52North 52North https://sensorweb.demo.52north.org/52n-sps-2.0/sps?request=GetCapabilities Working
WCS 2.0.1 rasdaman rasdaman Not able to access ----
WCS 2.0.1 OSGeo MapServer https://ows.eox.at/cite/mapserver?service=wcs&version=2.0.1&request=getcapabilities Working
EO-WCS 1.0 EOX IT Services GmbH EOxServer https://ows.eox.at/cite/eoxserver/ows?service=wcs&version=2.0.1&request=getcapabilities The test wcs10 is failed with "Exception in extension function java.net.MalformedURLException: no protocol"
WMS 1.1.1 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wms111?service=WMS&request=GetCapabilities Working
WMS 1.1.1 OSGeo GeoServer Do not support HTTPS ----
WMS 1.1.1 interactive instruments GmbH XtraServer 4.11 https://services.interactive-instruments.de/ogc-reference/cite/wms?request=getcapabilities&service=wms&version=1.1.1 Working
WMS 1.3.0 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wms130?service=WMS&request=GetCapabilities Working
WMS 1.3.0 OSGeo GeoServer Do not support HTTPS ----
WMS 1.3.0 interactive instruments GmbH XtraServer 4.11 https://services.interactive-instruments.de/ogc-reference/cite/wms?request=getcapabilities&service=wms&version=1.3 Working
WMTS 1.0.0 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wmts100?service=WMTS&request=GetCapabilities Working
WFS 1.0.0 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wfs100?service=WFS&request=GetCapabilities Working
WFS 1.1.0 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wfs110?service=WFS&request=GetCapabilities Working
WFS 1.1.0 interactive instruments GmbH XtraServer 4.11 https://services.interactive-instruments.de/ogc-reference/simple110/wfs?request=getcapabilities&service=wfs Working
WFS 2.0.0 lat/lon deegree 3.4 https://cite.deegree.org/deegree-webservices-3.4-RC3/services/wfs200?service=WFS&request=GetCapabilities Working
WFS 2.0.0 interactive instruments GmbH XtraServer 4.11 https://services.interactive-instruments.de/ogc-reference/simple/wfs?request=getcapabilities&service=wfs Working
WFS 2.0.0 Avitech GmbH Avitech SWIM (AxL) 4.0 - WFS https://wfst.axl.aero/AxlRest/wfs?service=WFS&version=2.0.0&request=GetCapabilities Failed with javax.net.ssl.SSLHandshakeException: PKIX path building failed
dstenger commented 6 years ago

Analyse results

SOS 2.0 | 52North | 52North No SSL error can be detected. As the DCP URLs use HTTP protocol, test are not executed against HTTPS URLs. -> No need for further investigation.

EO-WCS 1.0 | EOX IT Services GmbH | EOxServer Described error cannot be reproduced. Please specify the error report. No SSL related error can be detected. However, there are many test failures.

WFS 2.0.0 | Avitech GmbH | Avitech SWIM (AxL) 4.0 - WFS No SSL related error can be detected. Instead, following error is occurring:

Conclusion

I cannot detect any SSL related error in any test suite. I propose to create issues for observations if needed and to close this issue afterwards as there are no problems with SSL secured resources in general.

@keshav-nangare Can you please take a look at my results?

keshavnangare commented 6 years ago

@dstenger

WFS 2.0.0 : https://wfst.axl.aero/AxlRest/wfs?service=WFS&version=2.0.0&request=GetCapabilities

We get the two error with wfs20 i.e 1] same as mentioned in the last comment and 2] Certificate error

Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

This issue occurred while testing on production, we can see this error in user session/error_log/ directory. Attached the log log.txt

dstenger commented 6 years ago

@keshav-nangare Thank you for the input.

Indeed, this failure is related to SSL. I created a new issue for that: https://github.com/opengeospatial/ets-wfs20/issues/99

So, in conclusion, this is the only SSL related error which will be dealt with in the newly created issue. Thus, this issue can be closed.