Cannot display timeseries on click with nc file

[rowanwins]

Migrated from TerriaMap https://github.com/TerriaJS/TerriaMap/issues/474

We have some climate data served using ncWMS protocol (using THREDDS) and also a version using Geoserver (ncWMS plugin), we would like to display the time series graph for a specific point on click on the map. Is this feature available and is there some conf we can take as an example? Many thanks

cc @davidedelerma

[rowanwins]

Are you able to share the config for your featureInfoTemplate or the whole catalog item @davidedelerma ?

[davidedelerma]

Hi @rowanwins, this is the relevant part of the catalog:

{
         "featureInfoTemplate":{
            "name":"Number {{value}}",
            "template":"\u003cp\u003eTestt.\u003c/p\u003e\u003cchart src=\"http://thredds:8080/thredds/ncss/csvs/ERA5_AirTemp_30yrAv.nc?var=t2m\u0026latitude=0\u0026longitude=90\u0026time_start=2010-01-01T00%3A00%3A00Z\u0026time_end=2010-12-31T00%3A00%3A00Z\u0026accept=geocsv\"\u003e\u003c/chart\u003e"
         },
         "layers":"t2m",
         "name":"ncWMS with multiple dimensions, including time, with initialTimeSource",
         "type":"wms",
         "url":"http://thredds:8080/thredds/wms/csvs/ERA5_AirTemp_30yrAv.nc?service=WMS\u0026version=1.3.0\u0026request=GetCapabilities"
      }

[davidedelerma]

Data for the chart look like this:

# dataset: GeoCSV 2.0
# delimiter: ,
# field_unit: ISO_8601,degrees_north,degrees_east,celsius
# field_types: datetime,double,double,double
# field_missing: ,,,
time,latitude,longitude,t2m
2010-01-01T00:00:00Z,0.0,90.0,26.71843130013008
2010-01-02T00:00:00Z,0.0,90.0,26.54888602477093
2010-01-03T00:00:00Z,0.0,90.0,26.673380625141565
2010-01-04T00:00:00Z,0.0,90.0,26.649962228541526
2010-01-05T00:00:00Z,0.0,90.0,26.77026752483911
2010-01-06T00:00:00Z,0.0,90.0,26.83743525897265
2010-01-07T00:00:00Z,0.0,90.0,26.90677518856427
2010-01-08T00:00:00Z,0.0,90.0,26.78079507015168
2010-01-09T00:00:00Z,0.0,90.0,26.889290082089296
2010-01-10T00:00:00Z,0.0,90.0,26.79629875998518
2010-01-11T00:00:00Z,0.0,90.0,26.77804478600031
2010-01-12T00:00:00Z,0.0,90.0,26.69044165682617
2010-01-13T00:00:00Z,0.0,90.0,26.606582217464982
2010-01-14T00:00:00Z,0.0,90.0,26.71718969361378
2010-01-15T00:00:00Z,0.0,90.0,26.85599138309315
2010-01-16T00:00:00Z,0.0,90.0,26.64649114885121
2010-01-17T00:00:00Z,0.0,90.0,26.606480030021377
2010-01-18T00:00:00Z,0.0,90.0,26.774776447515684
2010-01-19T00:00:00Z,0.0,90.0,26.719052346709955
2010-01-20T00:00:00Z,0.0,90.0,26.762873534138304
2010-01-21T00:00:00Z,0.0,90.0,26.907799577136355
2010-01-22T00:00:00Z,0.0,90.0,26.942748887211117
2010-01-23T00:00:00Z,0.0,90.0,26.920104006227454
2010-01-24T00:00:00Z,0.0,90.0,26.839271708710505
2010-01-25T00:00:00Z,0.0,90.0,26.9338098079182
2010-01-26T00:00:00Z,0.0,90.0,26.72288712398022
2010-01-27T00:00:00Z,0.0,90.0,26.83957505374566
2010-01-28T00:00:00Z,0.0,90.0,26.920543259436204
2010-01-29T00:00:00Z,0.0,90.0,26.867162828563156
2010-01-30T00:00:00Z,0.0,90.0,27.035826454541446

[rowanwins]

@davidedelerma your template has some funky encoding going on, are you able to paste a more readable version?

[rowanwins]

I think you basically want something like

<chart src="http://thredds:8080/thredds/ncss/csvs/ERA5_AirTemp_30yrAv.nc?var=t2m&latitude={{terria.coords.latitude}}&longitude={{terria.coords.longitude}}&time_start=2010-01-01T00%3A00%3A00Z&time_end=2010-12-31T00%3A00%3A00Z&accept=xml"></chart>

In particular you want to make use of

latitude={{terria.coords.latitude}}&longitude={{terria.coords.longitude}}

Hopefully that should generate a chart in the feature info box.

[davidedelerma]

@rowanwins sorry this happen as kunermetes automatically encode html in json when json is parsed in configMap, the clean version is below:

{
         "featureInfoTemplate":{
            "name":"Number {{value}}",
            "template":"<p>Testt.</p><chart src='http://thredds:8080/thredds/ncss/csvs/ERA5_AirTemp_30yrAv.nc?var=t2m&latitude=0&longitude=90&time_start=2010-01-01T00%3A00%3A00Z&time_end=2010-12-31T00%3A00%3A00Z&accept=geocsv'></chart>"
         },
         "layers":"t2m",
         "name":"ncWMS with multiple dimensions, including time, with initialTimeSource",
         "type":"wms",
         "url":"http://thredds:8080/thredds/wms/csvs/ERA5_AirTemp_30yrAv.nc?service=WMS\u0026version=1.3.0\u0026request=GetCapabilities"
      }

[rowanwins]

If you're not getting a chart I'm wondering if it's because of the comments at the top of that csv...

[davidedelerma]

Yes I thought about adding the latitude={{terria.coords.latitude}}&longitude={{terria.coords.longitude}} to sort the problem of having the chart on Click on the map but actually the main problem now is that the chart does not display as you said. The comment on top could be the problem, but this is the way subsetting protocol returns geocsv using THREDDS I was wondering if you used a different query to THREDDS to have the chart displayed for example and also if I need to specify the column I need to display

[davidedelerma]

In my case the column I want to display is t2m

[rowanwins]

try accept=csv, I think it's strips out the comments

[davidedelerma]

In that case the response is something like this:

time,latitude[unit="degrees_north"],longitude[unit="degrees_east"],t2m[unit="celsius"]
2010-01-01T00:00:00Z,0.0,90.0,26.71843130013008
2010-01-02T00:00:00Z,0.0,90.0,26.54888602477093
2010-01-03T00:00:00Z,0.0,90.0,26.673380625141565
2010-01-04T00:00:00Z,0.0,90.0,26.649962228541526
2010-01-05T00:00:00Z,0.0,90.0,26.77026752483911
2010-01-06T00:00:00Z,0.0,90.0,26.83743525897265
2010-01-07T00:00:00Z,0.0,90.0,26.90677518856427
2010-01-08T00:00:00Z,0.0,90.0,26.78079507015168
2010-01-09T00:00:00Z,0.0,90.0,26.889290082089296

and I get an error like this in Terria:

Unable to load CSV file

See the csv-geo-au specification for supported CSV formats.

CSVDataError: Illegal Quote [Row:1][Col:2]

[davidedelerma]

I guess it does not like the quotations in the brackets....

[rowanwins]

:( Are you using terria master branch or next branch?

[davidedelerma]

I am using TerriaMap, this is the link to our package.json : https://github.com/SatelliteApplicationsCatapult/TerriaMap/blob/master/package.json

    "terriajs": "7.11.5",
    "terriajs-catalog-editor": "^0.2.0",
    "terriajs-cesium": "1.68.0",
    "terriajs-schema": "latest",

[davidedelerma]

Maybe I am using the wrong version of Terria or the problem is in the actual config I am giving? :(

[rowanwins]

At the moment we're working on a major overhaul of lots of Terria. It'll have the same functionality, we're just re-writing the model layer to be more modern.

Within TerriaMap and TerriaJS you'll find branches called next. These are used in production for some sites, and hopefully by the end of the year we're hoping to merge this into master. We're also just prepping some documentation for the community to help the transition, by and large things will be the same, but there might be some catalog config that needs updating.

CSV parsing in the next branch is one of things that's overhauled.

So depending on how far along you're along then moving to next might be an option.

[davidedelerma]

Cool, thanks for the reccomandation, I will try moving to the branch next and see if that helps

[kring]

This feature was requested ages ago, and I still think it's a great idea: https://github.com/TerriaJS/terriajs/issues/1424

My comment at the bottom has an idea for how it can be implemented pretty cleanly in Terria.

[rowanwins]

Thanks for the background @kring !

There is a catalog converter here with a hosted app here https://catalog-converter.terria.io/ Word of caution that the converter might still have some quirks.

Also FYI we haven't quite landed support for multi-dimensional WMS in next yet, there is PR where we're discussing a few things so it shouldn't be far off https://github.com/TerriaJS/terriajs/pull/4219)

[davidedelerma]

@kring @rowanwins would skipping comment lines in the csv parser in lib/Map/TableStructure.js be a quick fix to display timeseries coming from ncss? This might work for the geocsv files coming from ncss (example above)

[rowanwins]

I don't think so, I think the more problematic thing is the column heading rows with the " symbols, we'd need to look at where things are breaking down in that parsing process

[davidedelerma]

@kring @rowanwins Looking at the code in csv.js and looking at the output from the debugger, there is :

  $.csv = {
    defaults: {
      separator: ",",
      delimiter: '"',
      headers: true
    },

so the " are seen as a delimiter as far as I can understand....

[davidedelerma]

I have substituted " with another delimiter like | and now I can visualize the chart

[davidedelerma]

[rowanwins]

Awesome, glad you got it sorted @davidedelerma !

[davidedelerma]

@rowanwins Would be a a small PR in which I substitute the " delimiter with | fine with you?

[rowanwins]

Yep a PR would be welcomed! I'd perhaps add it as a configurable option, defaulting to existing behaviour.