aodn / aodn-portal

AODN Open Geospatial Portal
https://portal.aodn.org.au/
GNU General Public License v3.0
21 stars 13 forks source link

AIMS Weather and Temperature logger data collections cannot be downloaded #2960

Open vfisaac opened 1 year ago

vfisaac commented 1 year ago

Steps to reproduce the issue

  1. Goto here
  2. Select stations, next, download as csv
  3. Goto here
  4. Select stations, next, download as csv

Expected behaviour

Data is downloaded

Actual behaviour

Errors in csv files : Illegal property name: geom for feature type aims:weather_data Illegal property name: geom for feature type aims:temperature_data

Screenshots.

What version of the aodn portal are you using?

App version: 4.42.104

Which browser are you using?

Chrome

Any extra information that might be useful in the debugging process.

Both metadata records are harvested from AIMS, with AIMS geoserver links. Weather Stations - here Temperature Loggers - here

craigrose commented 1 year ago

@atkinsn The metadata is wrong. The WFS linkage needs to have the name "weather_deployment" / "temperature_deployment" not *_data.

<mrd:onLine>
<cit:CI_OnlineResource>
<cit:linkage>
<gco:CharacterString>https://geoserver.apps.aims.gov.au/aims/ows</gco:CharacterString>
</cit:linkage>
<cit:protocol>
<gco:CharacterString>OGC:WFS-1.0.0-http-get-capabilities</gco:CharacterString>
</cit:protocol>
<cit:name>
<gco:CharacterString>aims:weather_data</gco:CharacterString>
</cit:name>
<cit:description>
<gco:CharacterString>Weather data</gco:CharacterString>
</cit:description>
<cit:function>
<cit:CI_OnLineFunctionCode codeList="http://standards.iso.org/iso/19115/resources/Codelists/cat/codelists.xml#CI_OnLineFunctionCode" codeListValue=""/>
</cit:function>
</cit:CI_OnlineResource>
</mrd:onLine>
atkinsn commented 1 year ago

Communicated this with AIMS Data Centre.

Reply: Looks like we are hitting a performance issue with the data sets and geoserver ( which AODN sources from )

There are currently ~290million weather station entries, and ~190million temperature logger entries, and it's just not able to deliver it performantly.

We will have to investigate another way to present this data via geoserver / aodn - potentially by year, or some other way of partitioning the data.

Time series explorer does a variation of this already, but don't think they interacts with the portal.

craigrose commented 1 year ago

Had a meeting with AIMS (Shannon). They will look into configuring the layers so we can consume them. Meanwhile this is blocked.