search

opengeospatial / sensorthings

The official web site of the OGC SensorThings API standard specification.
135 stars 29 forks source link

CSV result format - V1.1 Request #51

Open KathiSchleidt opened 6 years ago

KathiSchleidt commented 6 years ago

In order to make STA data more accessible, it would be valuable to provide a simple CSV result format

liangsteve commented 5 years ago

I totally agree that it's very useful to have CSV outputs. Can you please offer more details here? There are many many different ways to flatten the data model and provide CSV outputs. Thoughts? (originally posted here)

hylkevds commented 5 years ago

Other issues dealing with alternative result formats are:

sgrellet commented 5 years ago

Copy / Paste from the proposal made in the original post :

"Proposal from our workshop at BRGM with Hylke and Kathi and BRGM folks Rationale only traverse the single cardinalities

hylkevds commented 4 years ago

I made a write up for a CSV resultFormat extension at: https://github.com/INSIDE-information-systems/SensorThingsAPI/blob/master/CSV-ResultFormat/CSV-ResultFormat.md

Comments are welcome, it's not implemented yet.

hylkevds commented 4 years ago

I've implemented the CSV extension. If anybody want to try it out, here is a link to a demo service: https://ogctest.docker01.ilt-dmz.iosb.fraunhofer.de/v1.1/Observations?$resultFormat=CSV&$select=id,phenomenonTime,result&$expand=Datastream($select=id,unitOfMeasurement;$expand=ObservedProperty($select=id,name))

mbeaufils commented 4 years ago

Is it be possible to select other separators instead of ","? For example ";" or "TAB". The possible issue I anticipate is some looong description fields that would contain "," in it.

hylkevds commented 4 years ago

RFC4180 describes how things like commas, quotes and line breaks should be handled in CSV files. If you change the separator to anything else, you've just moved the problem to that other character, which is just as likely to be in the strings...

mbeaufils commented 4 years ago

Yes I see your point and know that RFC4180 recommends to put the commas between quotes for proper CSV export. Does the CSV extension automatically add those quotes while doing the export?

Here is for example a kind of value we can have. for "Lithology" : Calcaire gréseux très glauconieux, coquillé, plus ou moins friable, peu fracturé, beige verdâtre à gris verdâtre.

Most of the time we change the separator for CSV export to solve the issue (generally using TAB instead). So my proposal was to enable to define the separator. I never faced a case where one would use , ; and TAB at the same time.

hylkevds commented 4 years ago

Yes, barring bugs, the output will be RFC4180 compliant. We probably should explicitly mandate that in the spec.

sgrellet commented 4 years ago

+1, we should emphasize that in the spec. so far the RFC is just mentionned for how to handle properties and parameters fields

sgrellet commented 4 years ago

+1 the implementation seems ok. your example already contains nasty combinations :) . we just tried the following

hylkevds commented 4 years ago

I've updated the spec to explicitly state that the resulting CSV must conform to RFC4180.