search

pacificclimate / pdp

The PCIC Data Portal - Server software to run the entire web application
GNU General Public License v3.0
1 stars 2 forks source link

Spatial selection is broken on the raster portals #198

Closed jameshiebert closed 3 years ago

jameshiebert commented 3 years ago

On the raster portals, the code that updates the download URL seems to be broken. The spatial sub-select component of the download URL doesn't get updated, and the following error shows up in the console.

Uncaught TypeError: bbox is undefined
    getRasterNativeProj https://data.pacificclimate.org/portal/js/pdp_raster_map.js:130
    onBoxChange https://data.pacificclimate.org/portal/js/pdp_controls.js:562
    canada_ex_app https://data.pacificclimate.org/portal/js/canada_ex_app.js:85
    jQuery 3
    getNCWMSLayerCapabilities https://data.pacificclimate.org/portal/js/data-services.js:50
    jQuery 6
    getNCWMSLayerCapabilities https://data.pacificclimate.org/portal/js/data-services.js:44
    canada_ex_app https://data.pacificclimate.org/portal/js/canada_ex_app.js:81
    triggerEvent https://data.pacificclimate.org/portal/js/OL/OpenLayers-2.13.1.js:138
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:199
    jQuery 8
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:197
    each jQuery
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:185
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:189
    each jQuery
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:185
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:189
    each jQuery
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:185
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:189
    each jQuery
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:185
    getRasterAccordionMenu https://data.pacificclimate.org/portal/js/pdp_controls.js:213
    jQuery 6
    getRasterAccordionMenuData https://data.pacificclimate.org/portal/js/data-services.js:24
    getRasterAccordionMenu https://data.pacificclimate.org/portal/js/pdp_controls.js:212
    getRasterControls https://data.pacificclimate.org/portal/js/pdp_controls.js:256
    canada_ex_app https://data.pacificclimate.org/portal/js/canada_ex_app.js:56
    jQuery 11
pdp_raster_map.js:130:15
    getRasterNativeProj https://data.pacificclimate.org/portal/js/pdp_raster_map.js:130
    onBoxChange https://data.pacificclimate.org/portal/js/pdp_controls.js:562
    canada_ex_app https://data.pacificclimate.org/portal/js/canada_ex_app.js:85
    jQuery 3
    getNCWMSLayerCapabilities https://data.pacificclimate.org/portal/js/data-services.js:50
    jQuery 6
    getNCWMSLayerCapabilities https://data.pacificclimate.org/portal/js/data-services.js:44
    canada_ex_app https://data.pacificclimate.org/portal/js/canada_ex_app.js:81
    triggerEvent https://data.pacificclimate.org/portal/js/OL/OpenLayers-2.13.1.js:138
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:199
    jQuery 8
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:197
    each jQuery
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:185
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:189
    each jQuery
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:185
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:189
    each jQuery
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:185
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:189
    each jQuery
    generateMenuTree https://data.pacificclimate.org/portal/js/pdp_controls.js:185
    getRasterAccordionMenu https://data.pacificclimate.org/portal/js/pdp_controls.js:213
    jQuery 6
    getRasterAccordionMenuData https://data.pacificclimate.org/portal/js/data-services.js:24
    getRasterAccordionMenu https://data.pacificclimate.org/portal/js/pdp_controls.js:212
    getRasterControls https://data.pacificclimate.org/portal/js/pdp_controls.js:256
    canada_ex_app https://data.pacificclimate.org/portal/js/canada_ex_app.js:56
    jQuery

@rod-glover and @corviday is this related to the ncWMS upgrade?

Steps to reproduce:

  1. Go to the PRISM map here
  2. Observe the download URL. Looks something like this: https://data.pacificclimate.org/data/bc_prism/pr_mClimMean_PRISM_historical_19710101-20001231.nc.nc?climatology_bounds,pr[][0:1680][0:3241]&
  3. Select a rectangle on the map
  4. Observe the download URL. It's the same. One would expect the spatial component to change... e.g. https://data.pacificclimate.org/data/bc_prism/pr_mClimMean_PRISM_historical_19710101-20001231.nc.nc?climatology_bounds,pr[][500:600][100:150]&
rod-glover commented 3 years ago

It seems likely that I broke it when I was making the new PRISM UI. I'll take this on.

rod-glover commented 3 years ago

In fact, it is broken because ncWMS 2.x has significantly different responses for GetCapabilities and GetFeatureInfo requests.

There's a fairly easy workaround for GetCapabilities.

The GetFeatureInfo request no longer returns the iIndex and jIndex values. The PDP makes use of this (former) feature to convert the corners of the selection rectangle to raster indices, which in turn determine the x-range and y-range in the Download URL.

I spent a fair bit of time experimenting with GetFeatureInfo, GetCapabilities, and GetMetadata requests to ncWMS 2.x and have not found an equivalent. It looks as if they simply do not support converting lat-long locations to raster indices any more. Furthermore, I could not locate any request whose response includes the lengths of the spatial dimensions of the raster, from which we could probably compute raster indices ourselves (assuming a uniform grid). They provide some time axis info, but no spatial axis info.

At this point, I think we will have to change where we get this information from. There seem to be the following possibilities:

  1. OpENDAP: The DDS response includes the length of the requested axes. From this we could compute raster indices, assuming uniform grids. Question: Is this a valid assumption? Do we have or expect non-uniform grids?
  2. PDP backend: Several options:
    • Expand the metadata.json endpoint to return grid dimensions and/or values. An explicit list of grid (lat, lon) values would enable us to handle a non-uniform grid.
    • Add a new endpoint (or extend an existing one -- metadata.json, with its already existing request parameter suggests itself) to do the raster index computation for us.

The lowest effort option is 1, but depends on a possibly invalid assumption.

Option 2 is more effort but cleaner. I am not sure which of 2 I prefer, but off the cuff, I think the second.

rod-glover commented 3 years ago

An interim solution suggested by @matthewbenstead , is to spin up an instance of the old ncWMS-PCIC, and direct GetFeatureInfo requests there. Going ahead with that while a full solution is decided and developed. Oy vey.

rod-glover commented 3 years ago

We have two main options, each with some sub-options:

  1. Compute index on client side.
    1. Assume uniform grids.
      • This is not strictly true, but it might be for all the PDP datasets. Need to find out which datasets are nonuniform.
      • Get grid dimensions (rank) from DDS, and use values of dataset bounding box (which we already have) to compute indices of selection box corners.
    2. Don't assume uniform grids. Get lat, lon value arrays, and use more general computation to compute indices.
  2. Compute index on server side.
    1. Add new endpoint
    2. Expand existing endpoint (metadata.json?)

The client side option is perhaps clunkier ... but really only puts the same computation client side.

The server side options all involve updating pdp_util, which is more fraught because there are some currently incompatible (w.r.t., pcic_meta database) updates to it already merged to master.

Choosing client side.

rod-glover commented 3 years ago

We are planning to migrate pcic_meta soon. That would remove the difficulties with the server-side solution. Things can continue to work using the temporary fix until then. Therefore we can consider the server side solutions without concerns re pdp_util, which are substantial.

rod-glover commented 3 years ago

The present code calls a function requestIndex(x,y) that returns the indices of the specified point. This function in turn makes a GetFeatureInfo request. Both client-side and server-side solutions would replace this with a different request to the backend, but overall the scheme would not change much, only its details.

There are 2 grids associated with 1040 datasets (data file variables; data files) that do not have evenly spaced y (lat) grids. Any solution will have to deal with these cases. The client-side solution, in particular, will need to retrieve the lat grid from DDS (https://services.pacificclimate.org/data/downscaled_gcms/tasmax_day_BCCAQv2+ANUSPLIN300_CanESM2_historical+rcp26_r1i1p1_19500101-21001231.nc.ascii?lat) and compute the index from that (and not rely on a simpler computation based on evenly spaced grids). The server-side solution would need the same data, retrieved directly from the database.

On balance, I think the relative complexity between

still comes down in favour of the client side solution.

rod-glover commented 3 years ago

:facepalm: I already had a PR well on the way to the client-side fix.