create a generic OGC URL print handler

[tbonfort]

Reporter: tomkralidis Date: 2004/10/07 - 01:19

(I categorized this under component WMS Server, however it applies to all OGC
supported specifications)

Bug 438, Bug 262, Bug 936 and Bug 943 call for support of URL type metadata in
various OGC Capabilities/etc. documents.

Between and among the abovementioned bugs, it was suggested that we come up with
a generic approach and function for this which the abovementioned bugs can
leverage, to implement msOWSPrintURLType() in mapows.c

Below is an analysis of OGC specifications / version which MapServer supports
which are affected and can leverage msOWSPrintURLType():

After some thought, some issues with the unified approach to printing URLs of
OGC services.  Here is where each element stands w.r.t each spec supported by
MapServer, and version:

WMS
===

Service level online resource (i.e. in WMT_MS_Capabilities/Service/)
----------------------------------------------------------------

OnlineResource in WMS 1.0.0:

<OnlineResource>[SOME_URL]</OnlineResource>

OnlineResource in WMS 1.1.0:

<OnlineResource xmlns:xlink="http://www.w3.org/1999/xlink" xlink:type="simple"
xlink:href="[SOME_URL]"/>

OnlineResource in WMS 1.1.1: same as 1.1.0

Layer level MetadataURL
-----------------------

MetadataURL in WMS 1.0.0: not available

MetadataURL in WMS 1.1.0:

<MetadataURL type="FGDC">
 <Format>text/html</Format>
 <OnlineResource xmlns:xlink="http://www.w3.org/1999/xlink"
xlink:href="[SOME_URL]"/> </MetadataURL>

NB: all elements/attributes REQUIRED
NB: @type must be either "FGDC" or "TC211"

MetadataURL in WMS 1.1.1: same as WMS 1.1.0

Layer level DataURL
-------------------

DataURL in WMS 1.0.0:

<DataURL>[SOME_URL]</DataURL>

DataURL in WMS 1.1.0:

<DataURL>
 <Format>content/unknown</Format>
 <OnlineResource xmlns:xlink="http://www.w3.org/1999/xlink" xlink:type="simple"
xlink:href="[SOME_URL]"/> </DataURL>

NB: all elements/attributes REQUIRED

DataURL in WMS 1.1.1: same as WMS 1.1.0

Layer level LegendURL
---------------------

LegendURL in WMS 1.0.0:
<Style>
 <Name>default</Name>
 <Title>default</Title>
 <StyleURL>[SOME_URL]</StyleURL>
</Style>

NB: all elements and attributes REQUIRED

LegendURL in WMS 1.1.0:

<Style>
 <Name>default</Name>
 <Title>default</Title>
 <LegendURL width="16" height="16">
  <Format>image/gif</Format>
   <OnlineResource xmlns:xlink="http://www.w3.org/1999/xlink"
xlink:type="simple" xlink:href="[SOME_URL]"/>  </LegendURL> </Style>

NB: all elements and attributes REQUIRED

LegendURL in WMS 1.1.1: same as WMS 1.1.0

OTHER
-----

I left out OnlineResource in the Root Layer element, which is different from
both service and layer level metadata, as MapServer doesn't support it presently.

WFS
===

Service level OnlineResource
----------------------------

OnlineResource in WFS 1.0.0:

<OnlineResource>http://cgns.nrcan.gc.ca</OnlineResource>

Layer level MetadataURL
-----------------------

MetadataURL in WFS 1.0.0

<MetadataURL type="FGDC" format="XML">[SOME_URL]</MetadataURL>

NB: @type must be "FGDC" or "TC211"
NB: @format must be "XML", "SGML", "HTML"

WMC
===

General level
-------------

LogoURL in WMC 1.0.0:

<LogoURL format="text/xml" width="10" height="10">  <OnlineResource
xlink:type="simple" xlink:href="[SOME_URL]"/> </LogoURL>

DescriptionURL in WMC 1.0.0:

<DescriptionURL format="text/html">
 <OnlineResource xlink:type="simple" xlink:href="[SOME_URL]"/> </DescriptionURL>

Layer level
-----------

Server in WMC 1.0.0:

<Server service="OGC:WMS" version="1.1.1" title="My OGC:WMS">  <OnlineResource
xlink:type="simple" xlink:href="[SOME_URL]"/> </Server>

MetadataURL in WMC 1.0.0:

<MetadataURL format="text/xml">
 <OnlineResource xlink:type="simple" xlink:href="[SOME_URL]"/> </MetadataURL>

DataURL in WMC 1.0.0:

<DataURL format="text/xml">
 <OnlineResource xlink:type="simple" xlink:href="[SOME_URL]"/> </DataURL>

LegendURL in WMC 1.0.0:
<Style>
 <Name>default</Name>
 <Title>default</Title>
 <LegendURL width="16" height="16">
  <Format>image/gif</Format>
   <OnlineResource xmlns:xlink="http://www.w3.org/1999/xlink"
xlink:type="simple" xlink:href="[SOME_URL]"/>  </LegendURL> </Style>

WCS
===

Service level metadata
----------------------

metadataLink in WCS 1.0.0:

<metadataLink metadataType="FGDC" xlink:type="simple" xlink:href="[SOME_URL]"/>

NB: @metadataType must be "FGDC", "TC211", "other"

Layer level metadata
--------------------

metadataLink in WCS 1.0.0:

<metadataLink metadataType="FGDC" xlink:type="simple" xlink:href="[SOME_URL]"/>

NB: @metadataType must be "FGDC", "TC211", "other"

SLD
===

ExternalGraphic in SLD 1.0.0:

<Graphic>
 <ExternalGraphic>
  <OnlineResource xlink:type="simple" xlink:href="[SOME_URL]"/>
  <Format>image/png</Format>
 <ExternalGraphic>
</Graphic>

OTHER
-----
I left out some deep darker corners of SLD
 - RemoteOWS
 - SemanticTypeIdentifier

..as MapServer doesn't support presently.

Filter
======
No URL types in specification

OTHER NOTES
===========

- none of the above analysis takes into account Service URLs for WMS, WFS or
WCS, i.e.:

<DCP>
 <HTTP>
  <Get>
   <OnlineResource xlink:type="simple" xlink:href="[SOME_URL]"/>
  </Get>
  <Post>
....and so on

- there's help on the way.  The OGC Common Implementation Specification (OGC
Discussion Paper -- http://portal.opengis.org/files/?artifact_id=6324 ) is
positioned (when adopted) for OGC specs to define their metadata and other
'common' bits as per OGC Common.  OGC Common defines ows:OnlineResourceType.

[tbonfort]

Author: tomkralidis Date: 2004/10/07 - 01:26

Additional thoughts:

msOWSPrintURLType() inputs:

- OGC spec type (e.g. wms, wfs, wcs, etc.)
- version (e.g. 1.1.1)
- format (e.g. image/png, text/html, TC211)
- url (e.g. http://host/someurl.html)

if (format =~ /^image/), then we need to additionally require:

- width (e.g. 10)
- height (e.g. 10)

I'm not sure whether we need to create a URL object before passing to such a
function, or sift through value in passing lp->metadata

As some elements are required for valid XML to DTD/Schema, if the required
metadata isn't in the mapfile, we'll want to not print it out, as opposed to
producing a commented warning, etc.?

[tbonfort]

Author: tomkralidis Date: 2004/10/12 - 19:31

Comments on the attachment made in comment #2:

This is a first hack of the logic for msOWSPrintURLType().  This is standalone
code to flush out logic/rules, should get much better as this gets integrated
into the main source.

Comments, suggestions, etc. valued.

[tbonfort]

Author: dmorissette Date: 2004/10/14 - 17:53

Let's make sure we add this in time for 4.4 (me or Assefa)

[tbonfort]

Author: dmorissette Date: 2004/10/19 - 22:12

Assigned to Julien

[tbonfort]

Author: jlacroix Date: 2004/10/19 - 23:47

Tom, I thought of implementing something more generic than what you attached, so
we don't have to add exceptions in the function for each release of a new spec
or for each addition of a new element. IMO, the exceptions should be added in
the code instead of in the OWS "generic" function. What I thought was more
something more like the msOWSPrintEncodeMetadata function.

msOWSPrintURLType(stream,  &(layer->metadata),
                  "namespace", "metadata_to_search", 
                  action_if_not_found /*OWS_NOERR or OWS_WARN*/,
                  "Format of the tag like this:
<MetadataURL%s%s%s>%s%s</MetadataURL>\n" /*In order type, width, height, format,
href */, 
                  "type format: type=\"%s\"", 
                  "width format", "height", "href", 
                  true, true, true, true, true /*tell if each element are
mandatory for this URL to be printed.*/)

Then the function will get all the metadata (metadata_to_search and append
_format and _href and etc.), put them in their own format and then put them in
the URL string.

If some metadata contains wrong string, it will be outputed like that so in WMS
MetadataURL you can specify any type you want without validation of what is in
the mapfile. This is like that in the code already and IMO, it is the
respossability of the mapfile builder to check what's he puts in it's mapfile.

Any comment/suggestion/objection?

[tbonfort]

Author: dmorissette Date: 2004/10/20 - 00:11

Sounds good to me (don't forget to mark the bug assigned)

[tbonfort]

Author: tomkralidis Date: 2004/10/21 - 05:32

Sounds good to me.  Let's go with it and test.

[tbonfort]

Author: jlacroix Date: 2004/10/21 - 17:30

I implemented the msOWSPrintURLType function (not commited yet), but it does not
seems that the current mapserver code will benefit much from it. It seems to be
only useful in the mapcontext.c code. WMS, WFS and WCS does not contains support
for MetadataURL, DataURL and LegendURL. Does the goal of this bugs (and others
that depends on it) is to implement those new elements in the GetCapabilities of
all OGC services?

I just want to clarify my mind.

[tbonfort]

Author: tomkralidis Date: 2004/10/21 - 17:45

Julien,

Please see the analysis in the first message of this bug, which lays out all
possibilities, etc. for all OGC specs.

[tbonfort]

Author: dmorissette Date: 2004/10/21 - 18:30

That's correct Julien: the objective of the other bugs (see dependencies) is to
add this to the other specs where it's missing.

[tbonfort]

Author: jlacroix Date: 2004/10/22 - 17:49

Steve, I will made the proposed changes to WCS. Is it ok with you? Mainly I will
just add support for the metadatalink metadata. I wanted to let you know.

Now, for this bug, the LegendURL in WMS gives me some trouble. It is included in
a <Style> element. Should we also include the style support? What I mean is that
currently, the web map context support the legendurl inside a style in the
wms_style_<name_of_the_style>_legendurl_(width/height/format/href). Should we
support this also and also get the name/title of the style? and also print each
style available in the stylelist? 

Or should we just support the wms_legendurl metadata?

My guest would be to try to get the style and all information about it. What do
you think.

[tbonfort]

Author: tomkralidis Date: 2004/10/22 - 18:19

I think, for the OGC:WMS case, we print only one style, which we can call
'default' or something.  This is because MapServer OGC:WMS doesn't supply
multiple style offers.

[tbonfort]

Author: jlacroix Date: 2004/10/25 - 19:44

Here's the function definition:
int msOWSPrintURLType(FILE *stream, hashTableObj *metadata, 
                      const char *namespaces, const char *name, 
                      int action_if_not_found, const char *tag_format, 
                      const char *tag_name, const char *type_format, 
                      const char *width_format, const char *height_format, 
                      const char *urlfrmt_format, const char *href_format,
                      int type_is_mandatory, int width_is_mandatory, 
                      int height_is_mandatory, int format_is_mandatory, 
                      int href_is_mandatory, const char *default_type, 
                      const char *default_width, const char *default_height, 
                      const char *default_urlfrmt, const char *default_href, 
                      const char *tabspace)

It used everywhere we have any URL components in OGC specs. It search for
<metadataname>_[width/height/format/href/type]. If one of this all required
components are present, the XML tag will be printed. If one is missing (but not
all), a warning will be printed. If the metadatas are not there at all, a
warning will be printed if it is required. See the code for more details.

Marking as FIXED.

BTW, for the WMS legendurl, I check for wms_style and for
wms_style_<style's_name>_legendurl_href metadata, both are required to have a
legendurl.

[tbonfort]

Author: sdlime Date: 2004/10/25 - 21:19

Are we really going to use a 22 parameter function? Is it just me or does this 
seem like a potential source for a lot of bugs...

Steve

[tbonfort]

Author: jlacroix Date: 2004/10/25 - 22:32

How does a new struct sound? Something like owsurlObj. This way we will just
have 6 parameter and still support every functionalities. Otherwise I think that
we can remove 7 parameters and oblige user to always pass 5 %s in the format
string (if not the right number, it will dramatically crash).

Which one do you prefer?

This struct can contain:
const char *tag_format, 
const char *tag_name, const char *type_format, 
const char *width_format, const char *height_format, 
const char *urlfrmt_format, const char *href_format,
int type_is_mandatory, int width_is_mandatory, 
int height_is_mandatory, int format_is_mandatory, 
int href_is_mandatory, const char *default_type, 
const char *default_width, const char *default_height, 
const char *default_urlfrmt, const char *default_href, 
const char *tabspace

[tbonfort]

Author: tomkralidis Date: 2004/10/26 - 05:41

I just checked, and mapwfs.c seems to be ok, but mapwcs.c and mapwms.c have
problems.

mapwcs.c: line 347 spits out broken XML:

<metadataLinkother xlink:type="simple"/>

mapwms.c:

- doing a OGC:WMS 1.1.1 GetCapabilities gives me an incomplete XML file, choking
on the first layer which has MetadataURL or DataURL defined.

[tbonfort]

Author: sdlime Date: 2004/10/26 - 06:56

Julien: Man, I don't know. I'd like to hear what Dan thinks. My initial thinking
that some sort of a structure would make sense, especially if there is an
initialization function to set common values. I was working in the WCS code,
probably making another instance of the problem Tom mentions, and needed a crib
sheet to decipher those function calls.

Steve

[tbonfort]

Author: jlacroix Date: 2004/10/26 - 17:36

Steve: I reopen in waiting of Daniel's comment. I probably wonder why we need so
many parameter in his function. It's because the we need to print up to 5
different metadatas inside the tag. Some may be optional, so we need a mechanism
to include each metadata in a tag with their own format that may or may not be
there. So we have a tag format which will be <metadataLink%s%s%s%s%s/> and each
metadata have their own format that will be included only if this is required.
This is for 5 function parameters. 5 others are there to tell the function if
the metadata are mandatory or not to this tag. And 5 others to define a default
value for each metadata. This gives us 15 function parameters (7 others are
quite the same as in other OWSPrint... functions.).

Tom: I commited the change to the WCS code to use the default value correctly.
However, the WMS GetCapabilities 1.0.0 and 1.1.1 works perfectly for me, even in
valgrind. Do you have any hint to help us to reproduce this (testcase or at
least mapfile+configure options). 

[tbonfort]

Author: sdlime Date: 2004/10/26 - 23:04

There were 2 spots in the WCS code, did you hit them both or just the one Tom
mentioned? Let me know and I'll fix the other too.

Steve

[tbonfort]

Author: jlacroix Date: 2004/10/26 - 23:29

I think I have fixed both of them.

[tbonfort]

Author: tomkralidis Date: 2004/10/27 - 00:02

OK, here is status update after testing:

- WCS works fine
 - we should make sure this is documented
- WFS works fine
 - we should make sure this is documented, and that the "type" and "format" are
strict enumerations as per schema
- WMS gives me back truncated output, even after make clean ./configure, make
 - configure options:

./configure --with-freetype=/usr/local/bin/freetype-config
--with-zlib=/usr/local --with-png=/usr/lo
cal --with-jpeg=/usr/local --with-xpm=/usr/local --with-gd=/usr/local
--with-pdf=/usr/local --with-e
ppl --with-proj=/usr/local --with-gdal=/usr/local/bin/gdal-config
--with-ogr=/usr/local/bin/gdal-con
fig --with-postgis=/usr/local/pgsql/bin/pg_config --with-wfs --with-wcs
--with-wmsclient --with-wfsc
lient --without-tiff --with-curl-config=/usr/local/bin/curl-config

- mapfile: http://devgeo.cciw.ca/ms_tmp/test.map

[tbonfort]

Author: dmorissette Date: 2004/10/27 - 00:39

with respect to comment #19 (risk of problems with a 22 parameters function), I
agreed initially and then had a look at the code and don't really have anything
to suggest. At least this function has the benefit that all URL types will use
consistent metadata names all over the place. It's definitely hard to use the
first time for a new spec but once it's in place we shouldn't have to touch it
any more.

Tom: with respect to your comments about documenting all this, bug 992 was
created by Julien. You may want to keep an eye on it.

[tbonfort]

Author: tomkralidis Date: 2004/10/27 - 05:10

w.r.t. OGC:WMS LegendURL, since MapServer supports GetLegendGraphic, should we
not autogenerate the LegendURL href, i.e.

xlink:href="[wms_onlineresource]"?version=[version]&service=WMS&request=GetLegendGraphic&format=[format]&layer=[layer]

unless, of course, wms_style_<style's_name>_legendurl_href is set in the mapfile?

[tbonfort]

Author: dmorissette Date: 2004/10/27 - 07:11

Tom, I think the automatic generation of the URL should be a new bug. It would
have to be done outside of msOWSPrintURLType() before calling it.

[tbonfort]

Author: jlacroix Date: 2004/10/27 - 21:31

Tom I fixed the WMS output problem. This is now in CVS. I put the assigment of
the result buffer before the encoding of the value, so it seg. fault when
encoding many characters. 

Marking as FIXED. 

In respect to the number of function parameters, I think that if someone wants
to reduce the number of parameters, we should create a new bug. The base
funcitonality is here, so we can mark as FIXED. Also, the comment on the top of
the function have been updated, so it should be easier to understand (Thanks
Daniel).

[tbonfort]

Author: tomkralidis Date: 2004/10/27 - 22:27

Almost.

OnlineResource elements are printing out:

link:href="[url]"

instead of

xlink:href="[url]"

[tbonfort]

Author: jlacroix Date: 2004/10/27 - 22:46

OK. xlink:href is now printed. Thanks a lot Tom!