qgis / QGIS

QGIS is a free, open source, cross platform (lin/win/mac) geographical information system (GIS)
https://qgis.org
GNU General Public License v2.0
10.35k stars 2.98k forks source link

Secured WMS regression in 3.28 / 3.34?; service does not load but loads in earlier QGIS versions / other wms clients. #55984

Open geoawd opened 7 months ago

geoawd commented 7 months ago

What is the bug or the crash?

I am having issues loading a secured WMS service in QGIS 3.28.14 and 3.34.2 on both Windows and Mac OS.

The same secured layer loads without issues in QGIS 3.22.16 (and a 3.16 that I had access to), as well as the requests being fulfilled in a web browser and ArcPro.

The credentials used are correct and a configuration has been saved. When you load the layer in 3.28.14 or 3.34, you are continually prompted for your password and the host requires authorisation. Sometimes in 3.28/3.34 a request will load and some data will display but 90% of requests are met with host requires authorisation.

You can see the difference in the video below: 3.34 on the left (constant prompts for the password) and 3.22 on the right.

https://github.com/qgis/QGIS/assets/119129964/b1140275-3b0f-4cab-ab97-3c5d447c275b

I have tried this on multiple computers and the result is the same. If this is intended behaviour in 3.28/3.34 can anyone provide some advice on how to load a secured wms service that works in both older QGIS and other WMS clients?

Thanks, Alex

Steps to reproduce the issue

  1. Add a secured WMS service in my case this url:(https://services.spatialni.gov.uk/ogc/services/Basemaps/OSNIFusionBasemap/MapServer/WMSServer [It's not my service so I can't provide credentials]

  2. Create a configuration with username and password; save the configuration.

  3. Add the WMS service to the map

  4. The master password will add the layer but you will be continually prompted to enter the password - and this will be met with a host requires authorisation. Occasionally some map tiles will be loaded.

Versions

This issue is affecting a secured wms service on: Windows 10; 3.28.14 / 3.34 Mac OS 3.34.1

Writing this on Mac OS where the layer loads in 3.22 but not in 3.34.

QGIS version 3.34.1-Prizren QGIS code revision 133927424d9 Qt version 5.15.2 Python version 3.9.5 GDAL/OGR version 3.3.2 PROJ version 8.1.1 EPSG Registry database version v10.028 (2021-07-07) GEOS version 3.9.1-CAPI-1.14.2 SQLite version 3.35.2 PDAL version 2.3.0 PostgreSQL client version unknown SpatiaLite version 5.0.1 QWT version 6.1.6 QScintilla2 version 2.11.5 OS version macOS 12.6

Active Python plugins processing 2.12.99 grassprovider 2.12.99 db_manager 0.1.20 MetaSearch 0.3.6

The services loads without any issues in 3.22

QGIS version 3.22.16-Białowieża QGIS code revision 6f08e4d7b0
Qt version 5.14.2
Python version 3.8.7
GDAL/OGR version 3.2.1
PROJ version 6.3.2
EPSG Registry database version v9.8.6 (2020-01-22)
GEOS version 3.9.1-CAPI-1.14.2
SQLite version 3.31.1
PostgreSQL client version 12.3
SpatiaLite version 4.3.0a
QWT version 6.1.4
QScintilla2 version 2.11.4
OS version macOS 12.6
       

Active Python plugins DEMto3D | 3.6 Qgis2threejs | 2.7.1 processing | 2.12.99 sagaprovider | 2.12.99 grassprovider | 2.12.99 db_manager | 0.1.20 MetaSearch | 0.3.5

QGIS version 3.22.16-Białowieża QGIS code revision 6f08e4d7b0 Qt version 5.14.2 Python version 3.8.7 GDAL/OGR version 3.2.1 PROJ version 6.3.2 EPSG Registry database version v9.8.6 (2020-01-22) GEOS version 3.9.1-CAPI-1.14.2 SQLite version 3.31.1 PostgreSQL client version 12.3 SpatiaLite version 4.3.0a QWT version 6.1.4 QScintilla2 version 2.11.4 OS version macOS 12.6

Active Python plugins DEMto3D 3.6 Qgis2threejs 2.7.1 processing 2.12.99 sagaprovider 2.12.99 grassprovider 2.12.99 db_manager 0.1.20 MetaSearch 0.3.5

Supported QGIS version

New profile

Additional context

No response

geoawd commented 7 months ago

Just to add to the above, the headers are sent with requests that return 200 and 401. See below. These were two consecutive requests in 3.34 where the first returned a 200 and the second a 401.

SuccessfulRequest FailedRequest
elpaso commented 7 months ago

I tested current master and 3.34 with a local GeoServer HTTP/basic auth and I could not find any issue.

elpaso commented 7 months ago

... forgot to mention: I tested on Linux, this bug may be OS dependent even if I doubt it because you could see it in both windows and mac.

geoawd commented 7 months ago

Would anyone have a secured ESRI wms service that they could check this with?

weca-theo commented 7 months ago

I've got 3.22.8 and 3.34.3. If I open up a blank QGIS project, create a new connection to an ArcGIS Online hosted WMS layer in the 'Data Source Manager'/ 'WMS/WMTS', then click 'Connect':

On 3.22.8 the layer loads into the layer list in under 1 second. Then I can click 'Add' to add the layer to the map.

On 3.34.3 the layer loads into the layer list in 2-3 minutes (!). Same machine, same network environment, same WMS layer. The data source manager window becomes unresponsive even after the layer list appears. So something is tanking this WMS window in 3.34. Choosing another WMS layer to connect to works fine. Another thing I've noticed is the UI looks horrific in 3.34- blurry text, no antialiasing, blurry icons, small text (even though all UI settings have been set to match my 3.22 settings).

geoawd commented 7 months ago

What’s happening with 3.28? Is that the same as 3.34? Are you trying a secured wms layer?

weca-theo commented 7 months ago

I don't have 3.28.

What do you mean by 'secured WMS'? Is that a WMS that requires username/password creds to use? I don't think I have one of those to hand. Happy to help test on 3.22.8, 3.32.1 or 3.34.3 if you wanted to share the credentials with me privately (I'm UK public sector, under PGSA agreement, so can handle any OS premium data).

geoawd commented 7 months ago

I’d be interested if anyone has a secured wms served from ESRI infrastructure that they can check this with?

I can’t share the credentials but there’s little point in that as I’ve tested this on multiple installations, on a couple of domains, on both Mac OS and windows and the particular secured wms (actually two different ones served from same host) that I’m struggling with work fine in 3.16/3.22.16 on both windows and Mac but will not consistently in 3.28 and 3.34 (I’ve tried multiple point releases).

I’ll raise it again with the service provider but they’re saying the service is fine (and it is in other clients).

elpaso commented 7 months ago

If some can share the credentials with a developer he can check what's going on with a debugger.

elpaso commented 7 months ago

The server does not accept HTTP Basic Authentication but only Digest which is not supported by the QGIS basic authentication plugin.

I find it hard to believe that this was working in older QGIS versions.

I am turning this into a feature request.

geoawd commented 7 months ago

Hi Alessandro, Thanks for looking into this. Attached is the log file of this loaded in 3.16; I have that installed on this PC so I just generated that. That service also works without issue in 3.24. Thanks Alex

QGIS version 3.16.6-Hannover QGIS code revision bfd36fddc9 Compiled against Qt 5.11.2 Running against Qt 5.11.2 Compiled against GDAL/OGR 3.1.4 Running against GDAL/OGR 3.1.4 Compiled against GEOS 3.8.1-CAPI-1.13.3 Running against GEOS 3.8.1-CAPI-1.13.3 Compiled against SQLite 3.29.0 Running against SQLite 3.29.0 PostgreSQL Client Version 11.5 SpatiaLite Version 4.3.0 QWT Version 6.1.3 QScintilla2 Version 2.10.8 Compiled against PROJ 6.3.2 Running against PROJ Rel. 6.3.2, May 1st, 2020 OS Version Windows 10 (10.0) Active python plugins changeDataSource; group_transparency; postgis_geoprocessing; pstimeseries; quick_map_services; slyr_community; SplitPolygon-master; db_manager; MetaSearch; processing

On Mon, Feb 5, 2024 at 10:34 AM Alessandro Pasotti @.***> wrote:

The server does not accept HTTP Basic Authentication but only Digest which is not supported by the QGIS basic authentication plugin.

I find it hard to believe that this was working in older QGIS versions.

I am turning this into a feature request.

— Reply to this email directly, view it on GitHub https://github.com/qgis/QGIS/issues/55984#issuecomment-1926675607, or unsubscribe https://github.com/notifications/unsubscribe-auth/A4M4O3C6PE3NFQKFAUHF7WTYSCYT7AVCNFSM6AAAAABCJOQP4OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMRWGY3TKNRQG4 . You are receiving this because you authored the thread.Message ID: @.***>

elpaso commented 7 months ago

I am sorry but there is nothing I can do, QGIS can only handle Basic auth and the server does not accept it (or maybe it doesn't accept it consistently)

See the last header here:

curl -v 'https://services.spatialni.gov.uk/ogc/services/Basemaps/OSNIFusionBasemap/MapServer/WMSServer?SERVICE=WMS&REQUEST=GetCapabilities'
*   Trying 194.32.20.105:443...
* Connected to services.spatialni.gov.uk (194.32.20.105) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
[...]
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=*.spatialni.gov.uk
*  start date: Feb 21 00:00:00 2023 GMT
*  expire date: Feb 20 23:59:59 2024 GMT
*  subjectAltName: host "services.spatialni.gov.uk" matched cert's "*.spatialni.gov.uk"
*  issuer: C=US; O=DigiCert, Inc.; CN=RapidSSL Global TLS RSA4096 SHA256 2022 CA1
*  SSL certificate verify ok.
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /ogc/services/Basemaps/OSNIFusionBasemap/MapServer/WMSServer?SERVICE=WMS&REQUEST=GetCapabilities HTTP/1.1
> Host: services.spatialni.gov.uk
> User-Agent: curl/7.81.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 
< Cache-Control: private
< WWW-Authenticate: Digest realm="UserDatabaseRealm", qop="auth", nonce="1707131544793:34b4d9bfc871dd9f4c8d9125fb534a98", opaque="D771BEC0D6A2B5BF8C737D5C99A91502"

If you try to authenticate with basic auth with curl it fails (while it works just fine with Digest):

ale@blackhole ~/dev/QGIS (bugfix-gh53956-GetLayerVisibility-deadlock)$ curl -v 'https://services.spatialni.gov.uk/ogc/services/Basemaps/OSNIFusionBasemap/MapServer/WMSServer?SERVICE=WMS&REQUEST=GetCapabilities' -u "*********:***********"  --basic
*   Trying 194.32.20.105:443...
* Connected to services.spatialni.gov.uk (194.32.20.105) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
*[....]
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=*.spatialni.gov.uk
*  start date: Feb 21 00:00:00 2023 GMT
*  expire date: Feb 20 23:59:59 2024 GMT
*  subjectAltName: host "services.spatialni.gov.uk" matched cert's "*.spatialni.gov.uk"
*  issuer: C=US; O=DigiCert, Inc.; CN=RapidSSL Global TLS RSA4096 SHA256 2022 CA1
*  SSL certificate verify ok.
* Server auth using Basic with user '*************'
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /ogc/services/Basemaps/OSNIFusionBasemap/MapServer/WMSServer?SERVICE=WMS&REQUEST=GetCapabilities HTTP/1.1
> Host: services.spatialni.gov.uk
> Authorization: Basic ************************************
> User-Agent: curl/7.81.0
> Accept: */*
> 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 
< Cache-Control: private
< WWW-Authenticate: Digest realm="UserDatabaseRealm", qop="auth", nonce="1707131012418:2dbafe0379325eb4e8e1527108f7a449", opaque="D771BEC0D6A2B5BF8C737D5C99A91502"
< Content-Type: text/html;charset=utf-8
< Content-Language: en
< Content-Length: 669
< Date: Mon, 05 Feb 2024 11:03:32 GMT
< Set-Cookie: CookiePersist=!LAakfk1A01Lj29xCSgh2GD+ElYV*******************************==; path=/; Httponly; Secure
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< Set-Cookie: TS01f27618=017f41f17b525d34d27e68f06636d3*********************************; Path=/
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff