search

eblondel / ows4R

R Interface for OGC Web-Services (OWS)
https://eblondel.github.io/ows4R/
Other
36 stars 8 forks source link

`describeCoverage` & `getCoverage` not returning all rangeType info for multiband coverages #88

Closed annakrystalli closed 1 year ago

annakrystalli commented 2 years ago

For the following RGB multiband coverage example, the rangeType object returned by DescribeCoverage only gives the details of the last band (BLUE).

library(ows4R)
#> Loading required package: geometa
#> Loading ISO 19139 XML schemas...
#> Loading ISO 19115 codelists...
#> Loading IANA mime types...
#> No encoding supplied: defaulting to UTF-8.
#> Loading required package: keyring
wcs <- ows4R::WCSClient$new(url = "https://geoserver.emodnet-physics.eu/geoserver/wcs",
                            serviceVersion = "2.0.1", logger = "DEBUG")
#> [ows4R][INFO] OWSGetCapabilities - Fetching https://geoserver.emodnet-physics.eu/geoserver/wcs?service=WCS&version=2.0.1&request=GetCapabilities
desc <- wcs$describeCoverage("emodnet__EP_GEO_NER_OTHR_NN_NN_RAS")
#> [ows4R][INFO] WCSClient - Fetching coverageSummary description for 'emodnet__EP_GEO_NER_OTHR_NN_NN_RAS' ... 
#> [ows4R][INFO] WCSDescribeCoverage - Fetching https://geoserver.emodnet-physics.eu/geoserver/wcs?service=WCS&version=2.0.1&coverageId=emodnet__EP_GEO_NER_OTHR_NN_NN_RAS&request=DescribeCoverage
#> Loading required package: sf
#> Linking to GEOS 3.9.1, GDAL 3.4.0, PROJ 8.1.1; sf_use_s2() is TRUE
# Only last band details are returned
desc$rangeType
#> <GMLElement>
#> ....|-- DataRecord <GMLElement>
#> ........|-- field <GMLElement>
#> ............|-- Quantity <GMLElement>
#> ................|-- description <GMLElement>
#> ....................|-- value: BLUE_BAND
#> ................|-- uom <GMLElement>
#> ................|-- constraint <GMLElement>
#> ....................|-- AllowedValues <GMLElement>
#> ........................|-- interval <GMLElement>
#> ............................|-- value: 0 255
# getRange() returns list of length 0
desc$getRange()
#> list()

Created on 2022-08-22 by the reprex package (v2.0.1)

Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.1.3 (2022-03-10) #> os macOS Big Sur/Monterey 10.16 #> system x86_64, darwin17.0 #> ui X11 #> language (EN) #> collate en_GB.UTF-8 #> ctype en_GB.UTF-8 #> tz Europe/Athens #> date 2022-08-22 #> pandoc 2.18 @ /Applications/RStudio.app/Contents/MacOS/quarto/bin/tools/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> askpass 1.1 2019-01-13 [1] CRAN (R 4.1.0) #> assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.1.0) #> class 7.3-20 2022-01-16 [1] CRAN (R 4.1.3) #> classInt 0.4-7 2022-06-10 [1] CRAN (R 4.1.2) #> cli 3.3.0 2022-04-25 [1] CRAN (R 4.1.2) #> codetools 0.2-18 2020-11-04 [1] CRAN (R 4.1.3) #> curl 4.3.2 2021-06-23 [1] CRAN (R 4.1.0) #> DBI 1.1.3 2022-06-18 [1] CRAN (R 4.1.2) #> digest 0.6.29 2021-12-01 [1] CRAN (R 4.1.0) #> dplyr 1.0.9 2022-04-28 [1] CRAN (R 4.1.2) #> e1071 1.7-11 2022-06-07 [1] CRAN (R 4.1.2) #> evaluate 0.15 2022-02-18 [1] CRAN (R 4.1.2) #> fansi 1.0.3 2022-03-24 [1] CRAN (R 4.1.2) #> fastmap 1.1.0 2021-01-25 [1] CRAN (R 4.1.0) #> fs 1.5.2 2021-12-08 [1] CRAN (R 4.1.0) #> generics 0.1.2 2022-01-31 [1] CRAN (R 4.1.2) #> geometa * 0.6-6 2022-01-26 [1] CRAN (R 4.1.2) #> glue 1.6.2 2022-02-24 [1] CRAN (R 4.1.2) #> highr 0.9 2021-04-16 [1] CRAN (R 4.1.0) #> htmltools 0.5.3 2022-07-18 [1] CRAN (R 4.1.2) #> httr 1.4.3 2022-05-04 [1] CRAN (R 4.1.2) #> jsonlite 1.8.0 2022-02-22 [1] CRAN (R 4.1.2) #> KernSmooth 2.23-20 2021-05-03 [1] CRAN (R 4.1.3) #> keyring * 1.3.0 2021-11-29 [1] CRAN (R 4.1.0) #> knitr 1.39 2022-04-26 [1] CRAN (R 4.1.2) #> lifecycle 1.0.1 2021-09-24 [1] CRAN (R 4.1.0) #> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.1.2) #> openssl 2.0.2 2022-05-24 [1] CRAN (R 4.1.2) #> ows4R * 0.3 2022-08-17 [1] Github (eblondel/ows4R@1cd9dbc) #> pillar 1.8.0 2022-07-18 [1] CRAN (R 4.1.2) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.1.0) #> proxy 0.4-27 2022-06-09 [1] CRAN (R 4.1.2) #> purrr 0.3.4 2020-04-17 [1] CRAN (R 4.1.0) #> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.1.2) #> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.1.2) #> R.oo 1.25.0 2022-06-12 [1] CRAN (R 4.1.2) #> R.utils 2.12.0 2022-06-28 [1] CRAN (R 4.1.2) #> R6 2.5.1 2021-08-19 [1] CRAN (R 4.1.0) #> Rcpp 1.0.9 2022-07-08 [1] CRAN (R 4.1.2) #> reprex 2.0.1 2021-08-05 [1] CRAN (R 4.1.0) #> rlang 1.0.4 2022-07-12 [1] CRAN (R 4.1.2) #> rmarkdown 2.14 2022-04-25 [1] CRAN (R 4.1.2) #> rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.1.0) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.1.0) #> sf * 1.0-7 2022-03-07 [1] CRAN (R 4.1.2) #> stringi 1.7.6 2021-11-29 [1] CRAN (R 4.1.0) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.1.2) #> styler 1.7.0 2022-03-13 [1] CRAN (R 4.1.2) #> terra 1.5-21 2022-02-17 [1] CRAN (R 4.1.2) #> tibble 3.1.8 2022-07-22 [1] CRAN (R 4.1.2) #> tidyselect 1.1.2 2022-02-21 [1] CRAN (R 4.1.2) #> units 0.8-0 2022-02-05 [1] CRAN (R 4.1.2) #> utf8 1.2.2 2021-07-24 [1] CRAN (R 4.1.0) #> vctrs 0.4.1 2022-04-13 [1] CRAN (R 4.1.2) #> withr 2.5.0 2022-03-03 [1] CRAN (R 4.1.2) #> xfun 0.31 2022-05-10 [1] CRAN (R 4.1.2) #> XML 3.99-0.10 2022-06-09 [1] CRAN (R 4.1.2) #> yaml 2.3.5 2022-02-21 [1] CRAN (R 4.1.2) #> #> [1] /Library/Frameworks/R.framework/Versions/4.1/Resources/library #> #> ────────────────────────────────────────────────────────────────────────────── ```

Using https://geoserver.emodnet-physics.eu/geoserver/wcs?service=WCS&version=2.0.1&coverageId=emodnet__EP_GEO_NER_OTHR_NN_NN_RAS&request=DescribeCoverage in the browser returns the full details

<wcs:CoverageDescriptions xmlns:wcs="http://www.opengis.net/wcs/2.0" xmlns:ows="http://www.opengis.net/ows/2.0" xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:gmlcov="http://www.opengis.net/gmlcov/1.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:swe="http://www.opengis.net/swe/2.0" xmlns:wcsgs="http://www.geoserver.org/wcsgs/2.0" xsi:schemaLocation=" http://www.opengis.net/wcs/2.0 http://schemas.opengis.net/wcs/2.0/wcsDescribeCoverage.xsd http://www.geoserver.org/wcsgs/2.0 https://geoserver.emodnet-physics.eu/geoserver/schemas/wcs/2.0/wcsgs.xsd">
<wcs:CoverageDescription gml:id="emodnet__EP_GEO_NER_OTHR_NN_NN_RAS">
<gml:description>Generated from WorldImage</gml:description>
<gml:name>Natural Earth features 5 types of raster files at 1:10 million-scale to suit your bandwidth and content focus. Two versions of the 10 million-scale raster data are offered: high resolution files at 21,600 x 10,800 pixels and low resolution at 16,200 x 8,100. The raster files register precisely with the 10m vector data. Embedded raster content includes: land cover, shaded relief, ocean water, and drainages with lakes. All files include a TFW world file.</gml:name>
<gml:boundedBy>
<gml:Envelope srsName="http://www.opengis.net/def/crs/EPSG/0/4326" axisLabels="Lat Long" uomLabels="Deg Deg" srsDimension="2">
<gml:lowerCorner>-90.000000000036 -180.0</gml:lowerCorner>
<gml:upperCorner>90.00000000000001 180.00000000007202</gml:upperCorner>
</gml:Envelope>
</gml:boundedBy>
<wcs:CoverageId>emodnet__EP_GEO_NER_OTHR_NN_NN_RAS</wcs:CoverageId>
<gml:coverageFunction>
<gml:GridFunction>
<gml:sequenceRule axisOrder="+2 +1">Linear</gml:sequenceRule>
<gml:startPoint>0 0</gml:startPoint>
</gml:GridFunction>
</gml:coverageFunction>
<gmlcov:metadata>
<gmlcov:Extension>
<ows:Keywords>
<ows:Keyword>WCS</ows:Keyword>
<ows:Keyword>WorldImage</ows:Keyword>
<ows:Keyword>NE1_HR_LC_SR_W_DR</ows:Keyword>
</ows:Keywords>
<ows:Metadata xlink:type="simple" xlink:href="https://catalogue.emodnet-physics.eu/geonetwork/srv/eng/xml_iso19139?uuid=bbeb010ad1d51578cd4a946f45eb7aea940a7e0d"/>
</gmlcov:Extension>
</gmlcov:metadata>
<gml:domainSet>
<gml:RectifiedGrid gml:id="grid00__emodnet__EP_GEO_NER_OTHR_NN_NN_RAS" dimension="2">
<gml:limits>
<gml:GridEnvelope>
<gml:low>0 0</gml:low>
<gml:high>21599 10799</gml:high>
</gml:GridEnvelope>
</gml:limits>
<gml:axisLabels>i j</gml:axisLabels>
<gml:origin>
<gml:Point gml:id="p00_emodnet__EP_GEO_NER_OTHR_NN_NN_RAS" srsName="http://www.opengis.net/def/crs/EPSG/0/4326">
<gml:pos>89.99166666666667 -179.99166666666667</gml:pos>
</gml:Point>
</gml:origin>
<gml:offsetVector srsName="http://www.opengis.net/def/crs/EPSG/0/4326">0.0 0.01666666666667</gml:offsetVector>
<gml:offsetVector srsName="http://www.opengis.net/def/crs/EPSG/0/4326">-0.01666666666667 0.0</gml:offsetVector>
</gml:RectifiedGrid>
</gml:domainSet>
<gmlcov:rangeType>
<swe:DataRecord>
<swe:field name="RED_BAND">
<swe:Quantity>
<swe:description>RED_BAND</swe:description>
<swe:uom code="W.m-2.Sr-1"/>
<swe:constraint>
<swe:AllowedValues>
<swe:interval>0 255</swe:interval>
</swe:AllowedValues>
</swe:constraint>
</swe:Quantity>
</swe:field>
<swe:field name="GREEN_BAND">
<swe:Quantity>
<swe:description>GREEN_BAND</swe:description>
<swe:uom code="W.m-2.Sr-1"/>
<swe:constraint>
<swe:AllowedValues>
<swe:interval>0 255</swe:interval>
</swe:AllowedValues>
</swe:constraint>
</swe:Quantity>
</swe:field>
<swe:field name="BLUE_BAND">
<swe:Quantity>
<swe:description>BLUE_BAND</swe:description>
<swe:uom code="W.m-2.Sr-1"/>
<swe:constraint>
<swe:AllowedValues>
<swe:interval>0 255</swe:interval>
</swe:AllowedValues>
</swe:constraint>
</swe:Quantity>
</swe:field>
</swe:DataRecord>
</gmlcov:rangeType>
<wcs:ServiceParameters>
<wcs:CoverageSubtype>RectifiedGridCoverage</wcs:CoverageSubtype>
<wcs:nativeFormat>image/tiff</wcs:nativeFormat>
</wcs:ServiceParameters>
</wcs:CoverageDescription>
</wcs:CoverageDescriptions>
annakrystalli commented 2 years ago

Hey @eblondel , any updates on this? Sorry to pester but at this point, this issue is holding up EMODnetWCS development.

eblondel commented 1 year ago

Hi @annakrystalli back from annual leave, i 'll look asap why multiple fields are not kept although not tackled as proper SWE bindings, and we get only one. I will create a ticket in geometa. It's a tricky issue but i'll do my best to solve it asap so you can keep going on EMODnetWCS devs.

Behind that, this deals with generic types defined geometa that depend on a standard (SWE - used essentially in SOS standard) that is not yet integrated in geometa (see https://github.com/eblondel/geometa/issues/187). Some recent tests were done, but it's not complete yet.

annakrystalli commented 1 year ago

Great! And thanks for the background info.

eblondel commented 1 year ago

@annakrystalli Please update geometa from Github. I've added the SWE DataRecord and fixed issue dealing with decoding. With this, you will get the rangeType as SWEDataRecord (from geometa binding), including the lit of fields.

Code:

require(ows4R)
wcs <- ows4R::WCSClient$new(url = "https://geoserver.emodnet-physics.eu/geoserver/wcs", serviceVersion = "2.0.1", logger = "DEBUG")
desc <- wcs$describeCoverage("emodnet__EP_GEO_NER_OTHR_NN_NN_RAS")
desc$rangeType

Output:

<SWEDataRecord>
....|-- field <SWEQuantity>
........|-- description: RED_BAND
........|-- uom[code=W.m-2.Sr-1]
........|-- constraint: 0 255
........|-- value: NA
....|-- field <SWEQuantity>
........|-- description: GREEN_BAND
........|-- uom[code=W.m-2.Sr-1]
........|-- constraint: 0 255
........|-- value: NA
....|-- field <SWEQuantity>
........|-- description: BLUE_BAND
........|-- uom[code=W.m-2.Sr-1]
........|-- constraint: 0 255
........|-- value: NA
annakrystalli commented 1 year ago

Awesome! Thanks @eblondel !

annakrystalli commented 1 year ago

Just looking into this today and although I can now get info on all bands, I cannot get nilValues anymore:

library(ows4R)
#> Loading required package: geometa
#> Loading ISO 19139 XML schemas...
#> Loading ISO 19115 codelists...
#> Loading IANA mime types...
#> No encoding supplied: defaulting to UTF-8.
#> Loading required package: keyring
wcs <- WCSClient$new(url = "https://geo.vliz.be/geoserver/Emodnetbio/wcs",
                            serviceVersion = "2.0.1")
summary <- wcs$getCapabilities()$findCoverageSummaryById("Emodnetbio__aca_spp_19582016_L1", exact = TRUE)
summary$getDescription()$rangeType
#> Loading required package: sf
#> Linking to GEOS 3.10.2, GDAL 3.4.2, PROJ 8.2.1; sf_use_s2() is TRUE
#> <SWEDataRecord>
#> ....|-- field <SWEQuantity>
#> ........|-- description: relative_abundance
#> ........|-- uom[code=W.m-2.Sr-1]
#> ........|-- constraint: -3.4028235E38 3.4028235E38
#> ........|-- value: NA

Created on 2022-09-29 with reprex v2.0.2

nilValue is returned by the server though:

https://github.com/EMODnet/EMODnetWCS/blob/ca4ffe9603381112654e249a33c97bf9872e0259/tests/fixtures/biology-description/geo.vliz.be/geoserver/Emodnetbio/wcs-206fed.xml#L756-L775

annakrystalli commented 1 year ago

BTW do you know when the dependency of ows4R on the corrected version of geometa will be formalised, either here or on CRAN?

Currently if someone installs EMODnetWCS (or latest dev version of ows4R from GitHub which EMODnetWCS currently requires) they do not get the right version of geometa installed by default. I tried adding the dev version of geometa as an explicit dependency of EMODnetWCS but R CMD CHECK complains with a note because geometa is not actually used in code in EMODnetWCS.

Any chance you could formalise the dependency on the dev version of geometa in the dev version of ows4R until they are both updated on CRAN?

eblondel commented 1 year ago

@annakrystalli Sorry for that you are right. Doing it right now. I will also look into the nilValues problem.

eblondel commented 1 year ago

@annakrystalli For the nilValues can you send me an example of coverage description concerned by that? thanks

annakrystalli commented 1 year ago

Sorry, how do you mean? The example I gave should be retuning the nilValue (it did with the previous version and I had written a function to extract it: https://emodnet.github.io/EMODnetWCS/reference/emdn_get_bbox.html) but isn't at all anymore so no way to adjust the function as I did for band descriptions, uoms etc. Here's the relevant section of the server response to the getDescription() request: https://github.com/EMODnet/EMODnetWCS/blob/ca4ffe9603381112654e249a33c97bf9872e0259/tests/fixtures/biology-description/geo.vliz.be/geoserver/Emodnetbio/wcs-206fed.xml#L756-L775

eblondel commented 1 year ago

Yes thanks for the xml example. nilValues here are yet another binding not yet added in geometa... I will have to support it there

eblondel commented 1 year ago

@annakrystalli ok you can reinstall geometa from github. I've enabled the decoding of nilvalues. SWE is still at experimental by the way... Normally you should get something like

<SWEDataRecord>
....|-- field <SWEQuantity>
........|-- description: relative_abundance
........|-- nilValues <SWENilValues>
............|-- nilValue: 9.969209968386869E36
........|-- uom[code=W.m-2.Sr-1]
........|-- constraint: -3.4028235E38 3.4028235E38
........|-- value: NA
annakrystalli commented 1 year ago

excellent thank you! I'll test it out now

eblondel commented 1 year ago

@annakrystalli SWE support in geometa is still experimental, so there may be changes in the way SWE objects are modelled. SWE introduces some more challenges in geometa to support generic encoding/decoding of XML. for WCS support we are more focusing on decoding, but for next needs in ows4R (in particular for SOS support) the SWE encoding is key.

In particular for the above, the nilValue will include a value field and not give the direct decimal value. For this I need to rely on simple SWE Element that is required to be able to decode/encode properly some fields such as nilValues. Similar logic had to be implemented to support GML elements in the past.

As print mode, it will look like this:

<SWEDataRecord>
....|-- field <SWEQuantity>
........|-- description  
............|-- value: relative_abundance
........|-- nilValues  <SWENilValues>
............|-- nilValue [reason=http://www.opengis.net/def/nil/OGC/0/unknown]
................|-- value: 9.969209968386869E36
........|-- uom [code=W.m-2.Sr-1] 
........|-- constraint  
............|-- AllowedValues  
................|-- interval  
....................|-- value: -3.4028235E38 3.4028235E38