search

opengeospatial / ets-security-client10

Public Repo for D112 Secure Client Tests
Other
1 stars 1 forks source link

How to mimic a server in a TestNG test suite #2

Open dstenger opened 6 years ago

openfirmware commented 6 years ago

Here is a rough idea of a secure client test implementation. Our goal is to test a secure client's ability to adhere to case 1 and case 2 of the Web Services Security Standard (I'm using document 17-007 as my reference).

Case 1 is to verify that the secure client can anonymously connect to the secure server using HTTPS and issue valid requests.

Case 2 is similar to Case 1, but the secure client must provide authentication to the server. The specific method of authentication is not defined, but it could be HTTP Basic or SSL Client Certificates.

Based on these cases and the existing workflow for the WMS Client Tests, I propose the following workflow.

  1. The Secure Client Test suite is installed into TEAM Engine the same way as other test suite modules.
  2. A tester who wants to validate their secure client then logs into the TEAM Engine instance in their web browser. They select the Secure Client Test from the test session options, and create the test session.
  3. In the TEAM Engine configuration options for the test session, the user selects the secure service type they want to emulate with the Secure Client Test. For example, they select "WPS" from a drop-down menu. They can also configure if they are testing Case 1 or Case 2.
  4. The user then creates the test session, and the Secure Client Test module creates a new HTTP/HTTPS server and registers a new URL handler for the test session id and secure service type. The URL could be something like https://localhost:8080/te/secure_client/s50/GetCapabilities. Based on the service type selected, a GetCapabilities template document is generated to emulate a bare-bones service response.
  5. The tester then uses this URL in their secure client, which will then open an HTTP connection to the Secure Client Test server.
  6. The Secure Client Test server then inspects the connection from the secure client and passes/fails the interaction based on tests. These results are printed out in the test session results in the web browser.
  7. The tester can review the test results to see any issues with their secure client.

I like that the tester can select the service type, which can then be emulated by spoofing a GetCapabilities document for that type of service. It may be necessary to respond to other requests such as GetMap — I'm not sure though. Doing it this way would make it easier to support more types of services by adding more templates.

When the tester is configuring the test session and selecting the conformance level (which use cases to validate), options for Case 2 could be "Require Any HTTP Basic parameters", "Require specific valid HTTP Basic parameters", or "Require SSL Client Certificate". This could be expanded to support other forms of authentication.

Something I am not sure about: Should the secure client test server provide an HTTP and HTTPS interface simultaneously? This way if the secure client only connects to the HTTP server and does not upgrade to the HTTPS interface, it can be marked as a failure. I didn't see any mention of HTTP Strict Transport Security (HSTS) in the WSS document, so I don't know if we can/should test that a secure client connecting to HTTP automatically switches to HTTPS.

As the Secure Client Test server provides an HTTPS interface, it needs a valid CA Certificate. This make setting up your own TEAM Engine instance a bit more work as you have to get an SSL certificate that your secure client will validate against its trust store (usually tied to the OS), or you have to generate a self-signed certificate and then manually have every secure client add that certificate to their trust store. The former case is easier for the OGC to handle as they already have a certificate for https://cite.opengeospatial.org/teamengine/, and someone who wants to test on their own TEAM Engine installation can probably get it to work with a free certificate from Let's Encrypt.

Another small note: When the Secure Client Test server registers a unique URL for the test session, it is probably more secure to use a random nonce instead of sequential test session ids. For example, https://localhost:8080/te/secure_client/ec86d722c7fd5dabb24c6e02508a4831/GetCapabilities. This would make it nearly impossible for two testers to accidentally (or intentionally) interfere with simultaneous testing sessions on the same TEAM Engine instance.

dstenger commented 6 years ago

Thank you for the detailed description of your approach.

However, I have a couple of general questions first:

openfirmware commented 6 years ago

The Web Services Security Standard (17-007) lists four conformance classes under section 6:

  1. Conformance Class "OWS Common"
  2. Conformance Class "WMS 1.1.1"
  3. Conformance Class "WMS 1.3.0"
  4. Conformance Class "Common Security"

I think we want to support all of these; on the thread WPS was a priority which would fall under OWS Common, so I will start with that one. This will give the secure client implementors something they can test against.

For the Abstract Test Suite, there is one under Annex B.4 for the client-side.

Client Requirement 11

For each security constraint found, the client SHALL construct valid and invalid requests. The service-side test harness will evaluate the requests and determine compliance.

I need to re-read the entire document to get a sense of which security constraints will apply. I think I can use JUnit tests to simulate these valid and invalid client requests against the TestNG classes.

dstenger commented 6 years ago

In general, your workflow is comprehensible and is a good starting point.

However, some details should be discussed / added:

openfirmware commented 6 years ago

I looked into how opengeospatial/ets-wms-client13 does their HTTP intercepts, and it looks like it uses AllocateMonitorUrl from TEAM Engine to create a servlet for each test session. The client then connects to that servlet URL and makes the sample requests, while the TEAM Engine test runner web interface must be kept open so an AJAX script can monitor the progress using the TEAM Engine REST API. This is a nice solution, as it uses some features built into TEAM Engine.

However, it means that wms-client13 does not support being ran in an IDE nor as a Java JAR because a TEAM Engine web interface is required. If we went with that solution for security-client10, then we too would be limited to only running in TEAM Engine. Additionally I am not fully sure of the scope of what the AllocateMonitorUrl functionality can do: can it check for HTTPS connections or if the user used authentication?

The other option is then to build a small HTTP server into security-client10, and have it start up for each test session. When a user starts a new test session, the following parameters could be passed into the test runner:

When using TEAM Engine the first two are configured using the web interface form and the rest are hard-coded or configured by the TEAM Engine administrator (letting web users specify what address/port a server starts on is probably a bad security idea).

When using an IDE or JAR, those parameters could be specified using test-run-props.xml. As the IDE or JAR user is trusted, all the parameters can be modified.

I will be working on adding this HTTP server into the code this week. My work-in-progress code is on a branch on my own fork, so we can review it before merging onto this main repository.

Regarding your points above:

Tests should be grouped in conformance classes

I agree, the Java package layout for ETS encourages this too. Above I mentioned the OGC Service Type parameter and Conformance Level parameter as separate items, perhaps instead they should be combined? For example, selecting "WMS 1.1.1" as your service type would make "Conformance Class WMS 1.1.1" active, "WMS 1.3.0" for "Conformance Class WMS 1.3.0", or selecting WPS/WFS/etc for "Conformance Class OWS Common". This means only one of the conformance classes can be selected at a time, unlike some other test suites.

I also wrote above that "Conformance Class Common Security" was a separate class, but it would be better to describe it as the base class from which the other three classes inherit. So a user would be selecting one of those three classes and the test session would first verify "Common Security" level and then the specific conformance class level. For example

Selection of service type is fine. However, I don't think we should create scenarios like Case 1 and Case 2. There should be a single test execution for all cases.

Sure, I guess validating the use case level would be part of the base conformance class tests?

Is it always possible to construct valid and invalid requests (e.g. what is an invalid request for Case 1)?

I was thinking the test server could check if the client opens with a regular HTTP request or with an HTTPS request. If the client starts with a regular HTTP request, then only use case 0 applies, and use case 1 and above would be considered invalid.

Regarding SSL Client Certificates, I agree that this will make installation of test suite more complex. This should be kept as easy as possible.

It would be really helpful to have instructions on how to use public certificates from Let's Encrypt, as well as setting up self-signed or similar for private local testing where the host is not on the internet (which is a requirement for activating certificates from Let's Encrypt). LE has a guide on setting up local certs, which I will reference and re-use for this project's documentation.

openfirmware commented 6 years ago

I have reviewed version 1.0 of the OGC Web Services Security Standard and have some updates on the client testing procedure. In this newer version of the specification, Annex B contains definitions for normative conformance tests for the client. It mentions the limitation of the test harness (i.e. this ETS project) is that the ETS has no way of knowing if the secure client actually parses the security annotation enhanced capabilities document, and that must be done by the user testing the secure client.

There are some things the ETS can check that satisfy Annex B as well as cover the goals of Testbed 14. Note that the ETS will have to conform to Annex A, but as there is no test suite for that yet, the conformance will have to be manually checked for now.

Test Harness (ETS) must be capable to be configured with compliance for each Conformance Class as defined by this standard

The three conformance classes are "OWS Common", "WMS 1.1.1", and "WMS 1.3.0" and are configured in TEAM Engine by selecting the emulated service type, which automatically selects the conformance class. I may have to update this interface to make it very obvious which conformance class will apply. (For IDE/JAR users, this is defined in test-run-props.xml.) The effect of selecting the conformance class here will affect the contents of the Capabilities document the Test Harness will return to the secure client, and the security annotations that will be included.

It must also be possible to configure the test harness to produce annotated capabilities with any combination of Requirements Classes

These Requirements Classes specify things like HTTP/1.1 Methods, W3C CORS, and so on from Section 6.1 (Conformance class “Common Security”). I don't have this in the interface yet, perhaps it could be a set of checkboxes and/or radio buttons to enable them and thus affect the capabilities document sent to the secure client. These requirements classes will affect how the secure client is evaluated in certain situations.

I don't see how these affect test results though, aside from testing HTTPS and authentication methods.

For HTTPS, if a client does not use it then the client fails the client conformance test "HTTPS" and Use Case I.

For any authentication method — limited to one of "none", "Open API", "OpenID Connect", or "SAML2" — the client must use that auth method else it fails Use Case II (In the case of "None", Use Case II does not apply). Testbed 14 specified "OpenID Connect" as a mandatory test case; "None" must also be mandatory to support testing Use Case I.

Conformance Test “Working on Capabilities with no Content section”

I can see this test being included for supporting Use Case III and Use Case IV, where the first request for capabilities only retrieves a partial capabilities document, which includes specific constraints for access to the catalogue and/or data. This should eventually be included in the ETS, but is not a requirement for Testbed 14.

Summary

Test cases within scope of this ETS and Testbed 14:

Test cases within scope of this ETS, but not as a part of Testbed 14:

I will be focusing on the first list as a part of Testbed 14, but I realize that the Engineering Report will have to include documentation on how the ETS can support the second list items in the future.