Inconsistent representation of URLs in templates and GetCapabilites implementations

[GoogleCodeExporter]

This issue was submitted to Derrick by Bob Simons.  He raises a good point 
about our clarity in writing the documentation.  It is old, so we should check 
the current versions of 1) templates, 2) ncSOS RC8, 3) 52N v0.7.5 and alter the 
templates to be consistent.

This will sound like a minor, petty, difference, but it is a good example of an 
anomaly that forces the client software to deal with different types of servers 
differently (or at least have code to deal with different possibilities): In 
the GetCapabilities response from the 52N server
http://sensorweb.demo.52north.org/52nSOSv3.2.1/sos
for every ows:Operation/ows:DCP/ows:HTTP,
  every ows:Get URL includes a ? at the end
  every ows:Post URL doesn't have a ? at the end.
I don't know if this is correct. But it is reasonable.

However, the IOOS GetCapabilities template at
https://code.google.com/p/ioostech/source/browse/trunk/templates/Milestone1.0/SO
S-GetCapabilities.xml
just has SOS_SERVER for both all Get URLs and Post URLs.

Similarly, the GetCapabilities document from the NDBC SOS server at
http://sdf.ndbc.noaa.gov/sos/server.php?request=GetCapabilities&service=SOS
does not include a ? at the end of the ows:Get URLs.

Similarly, the GetCapabilities document from the NOS SOS server at
http://opendap.co-ops.nos.noaa.gov/ioos-dif-sos/SOS?service=SOS&request=GetCapab
ilities
also does not include a ? at the end of the ows:Get URLs.

Original issue reported on code.google.com by dpsnowde...@gmail.com on 16 Jan 2014 at 6:31

[GoogleCodeExporter]

I completely agree with that our template XML's and representative endpoints 
(demo ncSOS and 52N) should be as consistent as humanly possible.

In the template GetCapabilities, the SOS_SERVER entries should be replaced with 
a fake but correct-looking URL's.

I don't know if the 52N approach is best, regarding the use of "?". Does OGC 
have anything to say about this, for SOS GetCapabilities, and more generally 
for other OGC GetCapabilities responses (since this is a "ows" element)? Is 
there a common practice used in, say, GeoServer GC's and other OGC W*S 
implementations? One place to ask may be the Python OWSLib list, to see what's 
most common, if OGC doesn't have a documented recommendation.

Original comment by emilioma...@gmail.com on 21 Jan 2014 at 8:54

[GoogleCodeExporter]

I did a brief google based survey of some example servers and more often than 
not I saw the URL punctuated with a ?.  Can't say whether it is in the spec or 
not yet.  

Original comment by dpsnowde...@gmail.com on 22 Jan 2014 at 10:15

[GoogleCodeExporter]

Section 3 'Syntax Components' of the RFC-6874 'Uniform Resource Identifier
(URI): Generic Syntax' reads:

The generic URI syntax consists of a hierarchical sequence of components
referred to as the scheme, authority, path, query, and  fragment.

      URI         = scheme ":" hier-part [ "?" query ] [ "#" fragment ]

Then, Section 3.4.  'Query' explains further:

   The query component is indicated by the first question mark ("?")
character and terminated by a number sign ("#") character or by the end of
the URI.

So, my interpretation of these rules is that "?" is a part of a query, and
as such should not be left behind if the query itself is omitted. Leaving
"?" alone without variables may be harmless (e.g. server may refresh the
page or do just nothing) but I believe that it is a bad practice.

Alex Birger
Systems Engineer

Skjei Telecom
7777 Leesburg Pike, Suite 315N
Falls Church, VA 22043
Direct:   +1 703 917 9889
Main:    +1 703 917 4077
Fax:����   +1 703 917 0098
e-mail: alex.birger@skjeitelecom.com
My availability: http://www.timebridge.com/mytime/alexanderbirger

CONFIDENTIALITY NOTICE: For intended Skjei Telecom recipients only. 

Original comment by alexande...@noaa.gov on 22 Jan 2014 at 11:24

[GoogleCodeExporter]

From OGC 06-121r9 OGC Web Services Common Standard [1] (from which SOS 
derives), section 11.2, page 82:

  A URL prefix is defined as a string including, in order, the scheme ("http" or "https"),
  Internet Protocol hostname or numeric address, optional port number, path, mandatory
  question mark '?', and optional string comprising one or more server-specific parameters
  ending in a mandatory ampersand '&'. Thus, an HTTP GET URL prefix always ends in
  either a question mark '?' or an ampersand '&'.

[1] http://portal.opengeospatial.org/files/?artifact_id=38867

Original comment by sh...@axiomalaska.com on 23 Jan 2014 at 4:29

[GoogleCodeExporter]

That's right, but... In the very same section it also says that "The '&' is
a separator between name/value pairs, and is therefore optional after the
last pair in the request string", which implies that "?" is also optional as
it is the same kind of separator (Table 35 on page 76). Moreover, in the
Table 34 the structure of the URL is presented as 

     http://host[:port]/path[?{name[=value]&}],

where "[ ]" denotes 0 or 1 occurrence of an optional part, and "{}" denotes
0 or more occurrences. That example places "?" in the optional query part,
which is in fact exactly the same as in the RFC 2616 that OGC spec refers
to. The RFC 2616 in section 3.2.2, page 18 provides the following structure
of the HTTP URL:

   http_URL = "http:" "//" host [ ":" port ] [ abs_path [ "?" query ]]

That syntactic record shows "?" as a part of an optional query rather than a
mandatory part of a URL.

Then, the RFC 2616 refers in turn to RFC 2396, which was replaced by RFC
3986 (sorry, I used a wrong RFC number in my previous comment), for syntax
details explanation. There (section 3.3, page 21) is a statement similar to
the one in the OGC spec:

      The path is terminated by the first question mark ("?") 
      or number sign ("#") character, or by the end of the URI.

However, the very next section 3.4, page 22, defines a query component as

      The query component is indicated by the first question
      mark ("?") character and terminated by a number sign ("#") character
      or by the end of the URI.

In addition, all RFCs and the OGC spec identify "?" as a general separator
along with "&" and "#".

Summing all that up, I believe that the statement in the OGC spec "HTTP GET
URL prefix always ends in either a question mark '?' or an ampersand '&'"
does not mean that "?" is a part of a prefix, and the URL must end up with
it even if query is missing. I believe that it rather means that "?"
separates prefix from query, and as such should be dropped altogether with
the query if the query is omitted.

Alex Birger
Systems Engineer

Skjei Telecom
7777 Leesburg Pike, Suite 315N
Falls Church, VA 22043
Direct:   +1 703 917 9889
Main:    +1 703 917 4077
Fax:����   +1 703 917 0098
e-mail: alex.birger@skjeitelecom.com
My availability: http://www.timebridge.com/mytime/alexanderbirger

CONFIDENTIALITY NOTICE: For intended Skjei Telecom recipients only. 

Original comment by alexande...@noaa.gov on 23 Jan 2014 at 5:52

[GoogleCodeExporter]

I'm sorry but I don't think there's any ambiguity here:

  "An Online Resource URL intended for HTTP GET requests is in fact only a URL prefix"

  "A URL prefix is defined as a string including...mandatory question mark '?'"

  "an HTTP GET URL prefix always ends in either a question mark '?' or an ampersand '&'"

It's true that the URL pattern in Table 39 indicates that the question mark is 
optional, but this is probably because they copied the pattern from another 
spec.

There are multiple instances of clear, deliberate language stating that the 
question mark is mandatory for GET URLs. I don't feel like there's any argument 
to be made unless there's something in the SOS spec itself to the contrary.

Original comment by sh...@axiomalaska.com on 23 Jan 2014 at 6:39

[GoogleCodeExporter]

Propose to close with following actions:

1. Add trailing ? to ows:Get URLs in IOOS GetCap template
2. Replace SOS_SERVER with fake but correct looking URL in IOOS GetCap template
3. Add ows:Get trailing ? to list of IOOS SOS rules

Original comment by sh...@axiomalaska.com on 23 Apr 2014 at 5:46

[GoogleCodeExporter]

+1 to Shane's 3 proposals.
#3 may be tricky b/c the documentation of IOOS SOS rules is in a state of flux, 
sort of. The official, up-to-date documentation is the monolithic Word/pdf 
document, but the more compartmentalized wiki page is still far easier to 
digest and edit, plus it's there. So, to minimize confusion both the wiki page 
and the official document should be updated.

Original comment by emilioma...@gmail.com on 23 Apr 2014 at 6:00