Expand the Sensor encodingType with a ValueCode for webpages/URLs

[EdwinWitt]

In

The encoding type for Sensors is currently limited to application / PDF and SensorML. I quote from the SensorThings API specification (http://docs.opengeospatial.org/is/15-078r6/15-078r6.html) "Most sensor manufacturers provide their sensor datasheets in a PDF format." However, there are also suppliers who place the sensor metadata on a web page. A concrete example that we have to deal with: https://www.dfrobot.com/wiki/index.php/Turbidity_sensor_SKU:_SEN0189

Is it possible that SensorThings is being expanded with a ValueCode for web pages (URLs)?

[liangsteve]

I think it's a good idea. I believe the proposal here is to add a new MIME type of html, i.e., text/html, to the ValueCode list?

That is adding the third row:

Sensor encodingType	ValueCode Value
PDF	application/pdf
SensorML	http://www.opengis.net/doc/IS/SensorML/2.0
an URL to a HTML page	text/html

Let's review it in our next meeting.

[EdwinWitt]

Correct!

When will this ValueCode be officially part of the SensorThings API specification?

Van: Steve Liang [mailto:notifications@github.com] Verzonden: maandag 5 februari 2018 16:41 Aan: opengeospatial/sensorthings CC: Witt, Edwin; Author Onderwerp: Re: [opengeospatial/sensorthings] Expand the Sensor encodingType with a ValueCode for webpages/URLs (#39)

I think it's a good idea. I believe the proposal here is to add a new MIME type of html, i.e., text/html, to the ValueCode list?

That is adding the third row: Sensor encodingType

ValueCode Value

PDF

application/pdf

SensorML

http://www.opengis.net/doc/IS/SensorML/2.0

an URL to a HTML page

text/html

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/opengeospatial/sensorthings/issues/39#issuecomment-363122589, or mute the threadhttps://github.com/notifications/unsubscribe-auth/Aie8ZFk8-JGyhfuB6Cs7_16bL9b2D0Ghks5tRyD4gaJpZM4R5YGt.

[hylkevds]

I think it would be better to allow any mime type, and only give a list of suggested types in the standard. That way we don't need to change the standard every time a new mime type is needed.

[KathiSchleidt]

Allow any mime type, or if no mime type is available a URL should be provided (ideally referencing a namespace). In Table 15, add "text/html"; through the reference above it becomes open And as the server doesn't care what's at the other end of the metadata reference, this has no impact on the server and can thus be left totally open

[selimnairb]

@KathiSchleidt @liangsteve @hylkevds As I sat down to write a PR I saw the need for clarification, specifically the definition of metadata in Table 13.

For instance if encodingType is application/pdf, then according the current description for metadata (i.e. "The detailed description of the Sensor or system. The metadata type is defined by encodingType."), the contents of metadata should be a PDF document. However that is a binary format, which the JSON encoding of course will not support, unless further encoded (e.g. base64; but how would a client know this without guessing?). However, this isn't how we are using metadata; we are instead storing a URL to a PDF document. So either we need to change the description of metadata to say that it is always a URL that refers to a resource of type encodingType, or we would need to revise the data model to allow for the direct storage of metadata of certain constrained types compatible with the JSON encoding. However this option is problematic because even something as simple as text/plain can include characters that are illegal for a JSON string.

So it seems to me that the intention is that metadata will always be a URL. Is this correct? If so, I would also like to revise the definition of metadata to reflect this (in addition to the changes described above).

[hylkevds]

Let's not make big changes for v1.1. If the entity returned is a JSON String, the client should be able to detect that it starts with http:// or https:// and thus be able to figure out it's a URL. If the entity returned is a JSON Object (containing JSON encoded SensorML) the client can also figure that out.

Lets only add the requested mime type(s) for v1.1 and some wording explaining the String-content-sniffing.

[selimnairb]

@hylkevds @taniakhalafbeigi @EdwinWitt @KathiSchleidt I created the PR to address this in 1.1.

[hylkevds]

Great, thanks!

[MarcusAlzona]

FYI the only mime type for URL/URI that I could find was from IETF RFC 2483:

https://tools.ietf.org/html/rfc2483#section-5

Published specification : Uniform Resource Locators (URLs) and
Uniform Resource Names (URNs) are two instances of the more general
class of identifiers known as Uniform Resource Identifiers (URIs).
URN resolution methods frequently wish to return lists of URLs for a
resource so that fault-tolerance and load balancing can be achieved.
The text/uri-list format is intended to be a very simple format for
communicating such lists of URLs (and URNs) in a form suitable for
automatic processing.

The format of text/uri-list resources is:

1) Any lines beginning with the '#' character are comment lines
    and are ignored during processing. (Note that URIs may contain
    the '#' character, so it is only a comment character when it is
    the first character on a line.)

2) The remaining non-comment lines shall be URIs (URNs or URLs),
    encoded according to the URL or URN specifications (RFC2141,
    RFC1738 and RFC2396). Each URI shall appear on one and only one
    line. Very long URIs are not broken in the text/uri-list format.
    Content-transfer-encodings may be used to enforce line length
    limitations.

3) As for all text/* formats, lines are terminated with a CRLF pair.

In applications where one URI has been mapped to a list of URIs, the
first line of the text/uri-list response SHOULD be a comment giving
the original URI.

Also referenced in https://www.iana.org/assignments/media-types/text/uri-list

How might (or how should) we utilize this?

[selimnairb]

@MarcusAlzona Thanks for looking into this. I could be wrong or missing something, but I'm not sure this would apple. I also ran into this, but it seemed to be constrained to lists of URIs. Now that you mention it again, I suppose a URI list could contain only one element. However, it seems that the purpose of the encodingType field is to indicate the type of the sensor metadata, not the type of the reference to the metadata. I updated the PR to change "resolvable link" to "URL", which is more specific and accurate I think. Do you think this suffices?

[KathiSchleidt]

Did I miss something in Banff? Thought we'd agreed to allow both variants as in @hylkevds posting above, so

• If the entity returned is a JSON String, the client should be able to detect that it starts with http:// or https:// and thus be able to figure out it's a URL.
• If the entity returned is a JSON Object (containing JSON encoded SensorML) the client can also figure that out.

I know its not semantically explicit, but do see strong use cases for both variants

[hylkevds]

Yes, that's what Brians PR currently describes.

[hylkevds]

Fixed with the publication of Sensing v1.1.