search

Esri / geoportal-server-catalog

Esri Geoportal Server is a next generation open-source metadata catalog and editor, based on elasticsearch.
https://www.esri.com/en-us/arcgis/products/geoportal-server/overview
Apache License 2.0
97 stars 60 forks source link

Error adding csw to client when using https and azure app gateway #472

Closed brehau closed 1 year ago

brehau commented 1 year ago

I have setup geoportal server catalog in AKS and are using azure application gateway as a reverse proxy. tls-termination happens on the app gateway. Elasticsearch is also running in the same cluster, and communication between that service and geoportal happens over https. When trying to add a csw-service to the metasearch-client in QGIS using this url: https://rasterportal-dev.miljodirektoratet.no/geoportal/csw , I get this error message:

Error connecting to service: Opening and ending tag mismatch: hr line 5 and body, line 6, column 8 (, line 6)

In the serverlogs, I get the following error:

│ java.net.SocketException: Unexpected end of file from server ││ at jdk.internal.reflect.GeneratedConstructorAccessor96.newInstance(Unknown Source) ~[?:?] ││ at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] ││ at java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[?:?] ││ at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1974) ~[?:?] ││ at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1969) ~[?:?] ││ at java.security.AccessController.doPrivileged(Native Method) ~[?:?] ││ at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1968) ~[?:?] ││ at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536) ~[?:?] ││ at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) ~[?:?] ││ at com.esri.geoportal.lib.elastic.http.ElasticClient.send(ElasticClient.java:194) ~[geoportal.jar:?] ││ at com.esri.geoportal.lib.elastic.http.ElasticClient.sendGet(ElasticClient.java:253) ~[geoportal.jar:?] ││ at com.esri.geoportal.lib.elastic.ElasticContextHttp.ensureIndex(ElasticContextHttp.java:131) [geoportal.jar:?] ││ at com.esri.geoportal.lib.elastic.ElasticContextHttp$1.run(ElasticContextHttp.java:257) [geoportal.jar:?] ││ at java.util.TimerThread.mainLoop(Timer.java:556) [?:?] ││ at java.util.TimerThread.run(Timer.java:506) [?:?] ││ Caused by: java.net.SocketException: Unexpected end of file from server ││ at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:900) ~[?:?] ││ at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:723) ~[?:?] ││ at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:897) ~[?:?] ││ at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:723) ~[?:?] ││ at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1615) ~[?:?] ││ at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520) ~[?:?] ││ at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:3114) ~[?:?] ││ at java.net.URLConnection.getContentType(URLConnection.java:519) ~[?:?] ││ at com.esri.geoportal.lib.elastic.http.ElasticClient.send(ElasticClient.java:180) ~[geoportal.jar:?]

I have tried running elasticsearch and geoportal locally in docker without https, and this works fine.

I noticed that the url's in the csw xml's i AKS all use http, so I have also tried changing the requestInfo.baseUrl to "https://rasterportal-dev.miljodirektoratet.no/geoportal" in \geoportal-search\src\main\resources\gs\context\nashorn\execute.js, and this changes the url's to https in the xml's but did not improve on the error.

Does anyone have any idèa if this might be a bug, or if not, what could be the issue here?

Thanks!

mhogeweg commented 1 year ago

hi, I checked the URL and it seems all operations URLs start with https. Perhaps you already solved the issue?

also, does the QGIS plugin support both 2.0.2 and 3.0.0 versions of CSW? for CSW 2.0.2, please test with this URL: https://rasterportal-dev.miljodirektoratet.no/geoportal/csw?request=GetCapabilities&service=CSW&version=2.0.2

brehau commented 1 year ago

Hi, and thanks for the quick response!

The urls in the dev-environmet are https because I hardcoded this in \geoportal-search\src\main\resources\gs\context\nashorn\execute.js as suggested in another issue here some time ago. This seemingly though, makes no difference for this issue. This url: https://rasterportal-test.miljodirektoratet.no/geoportal/csw?request=GetCapabilities&service=CSW&version=2.0.2 does not have this change, and uses http for the urls. I tried the full getCapabilities-url with the same result. It looks like the QGIS plugin supports the 2.0.2 version, but adding both the url you gave above and the one with just /csw wokrs locally (with domain substituted for localhost of course). I would therefore assume that the issue is related to the http-https difference. but in that case I would expect it to work with the changed dev-version of the site.

brehau commented 1 year ago

Hi, I ended up figuring this out. The issue was caused by owasp-rules in the Azure firewall. Adding exceptions for the relevant rules solved the issue.

Still, thanks for quick follow-up!