Testing a new resource with basic auth gives errors

[fsteggink]

When I'm adding a new WMS resource, specify "Basic Authentication" username + password, and then press the "Test" button at the Action row at the bottom of the page, I'm getting a result indicating that the test has failed. Details:

{
  "end_time": "2021-06-29T11:53:03Z",
  "message": "(, XMLSyntaxError('xmlParseEntityRef: no name, line 18, column 48'), )",
  "probes": [
    {
      "checks": [
        {
          "check_id": 9,
          "class": "GeoHealthCheck.plugins.check.checks.HttpStatusNoError",
          "message": "OK",
          "name": "HTTP status should not be errored",
          "response_time": -1,
          "success": true
        },
        {
          "check_id": 10,
          "class": "GeoHealthCheck.plugins.check.checks.XmlParse",
          "message": "(, XMLSyntaxError('xmlParseEntityRef: no name, line 18, column 48'), )",
          "name": "Valid XML response",
          "response_time": "0.167",
          "success": false
        },
        {
          "check_id": 11,
          "class": "GeoHealthCheck.plugins.check.checks.NotContainsOwsException",
          "message": "OK",
          "name": "Response NOT contains OWS Exception",
          "response_time": "0.14",
          "success": true
        },
        {
          "check_id": 12,
          "class": "GeoHealthCheck.plugins.check.checks.ContainsStrings",
          "message": "Title> not in response text",
          "name": "Response contains strings",
          "response_time": "0.13",
          "success": false
        }
      ],
      "class": "GeoHealthCheck.plugins.probe.owsgetcaps.WmsGetCaps",
      "message": "(, XMLSyntaxError('xmlParseEntityRef: no name, line 18, column 48'), )",
      "name": "WMS GetCapabilities",
      "probe_id": 3,
      "response_time": "0.170545",
      "success": false
    }
  ],
  "report_version": "1",
  "resource_id": 2,
  "resource_title": "Untitled",
  "resource_type": "OGC:WMS",
  "response_time": "0.177604",
  "start_time": "2021-06-29T11:53:03Z",
  "success": false,
  "url": "<my url>"
}

I was expecting a successful test result. When I save and close the resource, reopen it and then press "Test now", then the test passes successfully. Also when I press "Edit" and then the same "Test" button as in the beginning, the test succeeds. It is as if the credentials aren't used when the new check isn't saved to the database yet.

Also note that the resource title is "Untitled", although I had already given the new resource a proper name before I performed the test.

Output of docker logs ghc_web:

2021-06-29 13:52:39,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): test.omgevingsserver.nl:443
2021-06-29 13:52:39,808 - urllib3.connectionpool - DEBUG - https://<my server>:443 "GET /bgt/ows?service=WMS&request=GetCapabilities&version=1.3.0 HTTP/1.1" 302 277
2021-06-29 13:52:39,845 - urllib3.connectionpool - DEBUG - https://<my server>:443 "GET /login.html?httpd_location=<path>?service=WMS&request=GetCapabilities&version=1.3.0 HTTP/1.1" 200 1309
2021-06-29 13:52:39,847 - healthcheck - WARNING - Cannot use functools.partial(<function WebMapService at 0x7fececd8a440>, version='1.3.0') on <my url>: xmlParseEntityRef: no name, line 18, column 48 (<string>, line 18)
Traceback (most recent call last):
  File "/GeoHealthCheck/GeoHealthCheck/healthcheck.py", line 175, in sniff_test_resource
    ows = ows_handler(url)
  File "/venv/lib/python3.7/site-packages/owslib/wms.py", line 56, in WebMapService
    timeout=timeout, headers=headers, auth=auth)
  File "/venv/lib/python3.7/site-packages/owslib/map/wms130.py", line 75, in __init__
    self._capabilities = reader.read(self.url, timeout=self.timeout)
  File "/venv/lib/python3.7/site-packages/owslib/map/common.py", line 69, in read
    return etree.fromstring(raw_text)
  File "src/lxml/etree.pyx", line 3237, in lxml.etree.fromstring
  File "src/lxml/parser.pxi", line 1896, in lxml.etree._parseMemoryDocument
  File "src/lxml/parser.pxi", line 1784, in lxml.etree._parseDoc
  File "src/lxml/parser.pxi", line 1141, in lxml.etree._BaseParser._parseDoc
  File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
  File "<string>", line 18
lxml.etree.XMLSyntaxError: xmlParseEntityRef: no name, line 18, column 48
2021-06-29 13:52:39,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): <my server>:443
2021-06-29 13:52:39,972 - urllib3.connectionpool - DEBUG - https://<my server>:443 "GET /bgt/ows?service=WMS&request=GetCapabilities&version=1.1.1 HTTP/1.1" 302 277
2021-06-29 13:52:40,016 - urllib3.connectionpool - DEBUG - https://<my server>:443 "GET /login.html?httpd_location=/bgt/ows?service=WMS&request=GetCapabilities&version=1.1.1 HTTP/1.1" 200 1309
2021-06-29 13:52:40,019 - healthcheck - WARNING - Cannot use functools.partial(<function WebMapService at 0x7fececd8a440>, version='1.1.1') on <my url>: xmlParseEntityRef: no name, line 18, column 48 (<string>, line 18)
Traceback (most recent call last):
  File "/GeoHealthCheck/GeoHealthCheck/healthcheck.py", line 175, in sniff_test_resource
    ows = ows_handler(url)
  File "/venv/lib/python3.7/site-packages/owslib/wms.py", line 52, in WebMapService
    timeout=timeout, headers=headers, auth=auth)
  File "/venv/lib/python3.7/site-packages/owslib/map/wms111.py", line 75, in __init__
    self._capabilities = reader.read(self.url, timeout=self.timeout)
  File "/venv/lib/python3.7/site-packages/owslib/map/common.py", line 69, in read
    return etree.fromstring(raw_text)
  File "src/lxml/etree.pyx", line 3237, in lxml.etree.fromstring
  File "src/lxml/parser.pxi", line 1896, in lxml.etree._parseMemoryDocument
  File "src/lxml/parser.pxi", line 1784, in lxml.etree._parseDoc
  File "src/lxml/parser.pxi", line 1141, in lxml.etree._BaseParser._parseDoc
  File "src/lxml/parser.pxi", line 615, in lxml.etree._ParserContext._handleParseResultDoc
  File "src/lxml/parser.pxi", line 725, in lxml.etree._handleParseResult
  File "src/lxml/parser.pxi", line 654, in lxml.etree._raiseParseError
  File "<string>", line 18
lxml.etree.XMLSyntaxError: xmlParseEntityRef: no name, line 18, column 48
2021-06-29 13:52:40,020 - healthcheck - ERROR - Getting metadata failed: Cannot get OGC:WMS service instance for <my url>
Traceback (most recent call last):
  File "/GeoHealthCheck/GeoHealthCheck/healthcheck.py", line 183, in sniff_test_resource
    raise ValueError(message)
ValueError: Cannot get OGC:WMS service instance for <my url>

Note the line with https://<my server>:443 "GET /login.html?httpd_location=<path>?service=WMS&request=GetCapabilities&version=1.3.0 HTTP/1.1. I've set up my web server in such a way that when no authorization headers are provided the user is redirected to a login page. Obviously this is not a valid GetCapabilities document, hence the various kinds of errors (invalid XML and missing Title string).

[justb4]

It looks like you hit the Test button before saving? You first need to do Save and then Test. Otherwise the form is not saved. This may not be obvious from a UI perspective, improvement welcomed. Let's first see if that (not clicking Save first) was the reason.

[fsteggink]

I've just noticed that the first probe by ghc_runner fails as well. Perhaps this has the same cause. Too early to tell. I'll create a new issue if this has a different cause.

[fsteggink]

It looks like you hit the Test button before saving? You first need to do Save and then Test. Otherwise the form is not saved. This may not be obvious from a UI perspective, improvement welcomed. Let's first see if that (not clicking Save first) was the reason.

Save + Test results in the test being succeeded. :+1:

[fsteggink]

But the first probe run still fails (as reported only 2 mins ago)

[justb4]

Yes, this is a bit tricky: when adding a new Resource, we want users to specify an absolute minimum, only the endpoint URL. GHC then assigns and runs the default Probe for the endpoint type, usually a GetCapabilities Probe. This can even be overridden (GHC_PROBE_DEFAULTS), maybe you can make this empty (?), so that assignment/run is skipped (?).

In earlier versions the Resource was not added if that default Probe failed. But now it is added unconditionally. I can see that this is an issue for authenticated Resources. A solution would be to add auth-fields to the 'Add Resource' form.