Closed smnorris closed 9 months ago
Thanks @smnorris for the heads up. Just tested it now and it seems ok? We use it in listing the WFS records, and getting the single download limit, here.
Eg:
bcdata:::bcdc_get_wfs_records()
#> # A tibble: 861 × 3
#> whse_name title cat_url
#> <chr> <chr> <chr>
#> 1 WHSE_BASEMAPPING.DBM_BC_7H_MIL_BATHYMETRC_POLY (7.5M) Bathy… https:…
#> 2 WHSE_BASEMAPPING.DBM_BC_7H_MIL_BOUNDARIES_LINE (7.5M) Bound… https:…
#> 3 WHSE_BASEMAPPING.DBM_BC_7H_MIL_DRAINAGE_LINE (7.5M) Drain… https:…
#> 4 WHSE_BASEMAPPING.DBM_BC_7H_MIL_DRAINAGE_POLY (7.5M) Drain… https:…
#> 5 WHSE_BASEMAPPING.DBM_BC_7H_MIL_POPULATION_POINT (7.5M) Major… https:…
#> 6 WHSE_BASEMAPPING.DBM_BC_7H_MIL_ROADS_LINE (7.5M) Major… https:…
#> 7 WHSE_BASEMAPPING.DBM_BC_7H_MIL_PARK_POLY (7.5M) Natio… https:…
#> 8 WHSE_BASEMAPPING.DBM_BC_7H_MIL_POLITICAL_POLY (7.5M) Provi… https:…
#> 9 WHSE_LEGAL_ADMIN_BOUNDARIES.OATS_ALR_POLYS ALC ALR Poly… https:…
#> 10 WHSE_LEGAL_ADMIN_BOUNDARIES.OATS_ALR_BOUNDARY_LINES_SVW ALC Agricult… https:…
#> # ℹ 851 more rows
bcdata:::bcdc_single_download_limit()
#> [1] 10000
Created on 2023-08-22 with reprex v2.0.2
Glad it is still working.
I was using getCapabilities to:
~~1) Get the schema of a layer and find geometry column name
Schemas are no longer included in the response but I saw you use DescribeFeatureType
for that so I adjusted to do the same.~~ Edit - actually no, owslib
was doing DescribeFeaturetype internally, this didn't come from GetCapabilities
2) list WFS layers
I can adjust my code to use the current response to generate a list of layers as per bcdata:::bcdc_get_wfs_records()
But the current response does not seem to be a valid response to a WFS getcapabilites request and I don't think you'll be able to get the single download limit, the OperationsMetadata
tag is not present. I've alerted DataBC - https://github.com/smnorris/bcdata/issues/142#issuecomment-1686770997
I don't think you'll be able to get the single download limit, the
OperationsMetadata
tag is not present.
Weird. I can't find it manually but I stepped through bcdc_single_download_liimit()
and it is working (and not using a cached value).
(I fully admit though that I don't really understand xml parsing).
Weird indeed, I presumed it was picking up the default when I tried.
I don't want to parse xml either, and was using owslib
to do that for me. It bails when it cannot find OperationsMetadata
This is what the response used to look like: https://gist.githubusercontent.com/smnorris/57b7ff6df0354c4d13dab5130e55c8af/raw/682ce6d4c23cb670242e54bb6c3c6d829031d3c6/capabilities_wfs.xml
Alright, so this is the endpoint that our code is hitting: http://openmaps.gov.bc.ca/geo/pub/ows?service=WFS&version=2.0.0&request=Getcapabilities, and that has the information in it. It differs in two ways:
request=Getcapabilities
vs request=GetCapabilities
🤯
Oh dear. I think I standardized that url to https in my code, so there may not have been a change to the service?
Something definitely broke on Friday before I made changes but perhaps it was something else that has since been resolved.
Having a totally different response with http vs https is... weird? Also I would think it should all be https anyway!
And it does appear to be a combination of http AND the case of get[Cc]apabilities
. This returns WMS 1.1 info:
http://openmaps.gov.bc.ca/geo/pub/ows?service=WFS&version=2.0.0&request=GetCapabilities
And this appears to return nothing or time out:
https://openmaps.gov.bc.ca/geo/pub/ows?service=WFS&version=2.0.0&request=Getcapabilities
Based on my limited sysadmin experience, it sounds like an issue with reverse proxy setup.
It seems very fragile! Makes you wonder which is the intended endpoint? Hopefully the one we want ;)
Thanks for digging into this - I've forwarded the source of the issue and a link to this thread on to the DataBC JIRA ticket.
Awesome, thanks. If you get a resolution, do you mind updating this issue so we can update the url if needbe?
Will do.
I think the handling of the get[Cc]apabilites
casing did change, it still fails after I switch my base url to plain http.
owslib
generates the request as GetCapabilites
: https://github.com/geopython/OWSLib/blob/master/owslib/feature/common.py#L35
Thanks!
Interesting - looks like you could supply the request
param directly though as Getcapabilities
?
yes, but no luck so far - hopefully I can find the right combination of parameters.
My apologies - a quick fix was put in Aug 18 by Data Systems and Services to cache the WMS GetCapabilities request to support the delivery of wildfire services. The apache rewrite rule also erroneously captured WFS GetCapabilities requests. I removed these changes on Sept 6 - again, apologies for any impacts.
FYI, perhaps it is just a temporary outage but it seems like the response to a WFS
GetCapabilities
request has changed, and no longer returns info about the WFS servicehttps://openmaps.gov.bc.ca/geo/pub/ows?service=WFS&version=2.0.0&request=GetCapabilities https://github.com/smnorris/bcdata/issues/141
Python bcdata is more dependent on this response, but R package might be affected as well? Certainly when getting the
bcdata.single_download_limit