Reading-eScience-Centre / edal-java

Environmental Data Abstraction Layer libraries
Other
39 stars 30 forks source link

Extreme slowness with this UGRID dataset #45

Closed rsignell-usgs closed 8 years ago

rsignell-usgs commented 8 years ago

I added the OPeNDAP Data URL from this UGRID dataset http://comt.sura.org/thredds/dodsC/data/comt_1_archive/inundation_tropical/UND_ADCIRC/Hurricane_Ike_3D_final_run_with_waves.html and it took a very long time -- I snapped this picture after 6 hours: ncwms2_c

It eventually succeeded however, with no errors.

Now when I try to request an image (for example, of the water level variable "zeta"), using Godiva3 http://geoport.whoi.edu:8084/ncWMS2/Godiva3.html I only got a few tiles after several minutes of waiting.

And when I try the "test PNG image" for "zeta" from the ncWMS page: 2016-03-02_8-50-01

http://geoport.whoi.edu/ncWMS2/wms?REQUEST=GetMap&VERSION=1.3.0&STYLES=&CRS=CRS:84&WIDTH=256&HEIGHT=256&FORMAT=image/png&TRANSPARENT=true&LAYERS=ADCIRC-IKE-3D-Waves/zeta&BBOX=-97.856889,18.151389,-79.739305,31.010635000000004

I wait for 5 minutes and then get a 502 proxy error. 2016-03-02_8-50-01

The data is from a remote OPeNDAP endpoint, but this slowness seems extreme -- when I request an image from sci-wms, which runs on the same machine as ncWMS and access the data from same OPeNDAP endpoint, I get an image back in a few seconds:

http://sci-wms.whoi.edu/wms/datasets/adcirc_ike_3d_waves?time=2008-09-13T10%3A30%3A00Z&colorscalerange=-2.00%2C5.00&numcontours=40&styles=filledcontours_jet&version=1.1.1&srs=EPSG%3A3857&transparent=true&format=image%2Fpng&exceptions=application%2Fvnd.ogc.se_xml&tiled=true&feature_count=101&service=WMS&request=GetMap&layers=zeta&bbox=-10410111.756214725%2C3443946.746416902%2C-10331840.239250705%2C3522218.2633809224&width=256&height=256

guygriffiths commented 8 years ago

There's a couple of issues here, but the sticking point at the moment is related to the credentials - specifically I'm getting 403 Forbidden returned on all OPeNDAP requests.

OPeNDAP credential handling needed some reworking from ncWMS v1.x and was very low priority, so it's currently not implemented. That's what's causing the issue here. Are you either able to provide me with some login details for that OPeNDAP dataset or point me to wherever I can apply for some myself? Once I can access the data it should all work at a reasonable speed, after which I'll need to add some front-end configuration of credentials.

rsignell-usgs commented 8 years ago

That is strange -- the TDS serving the data is completely open.
Do you get a 403 on this? http://comt.sura.org/thredds/dodsC/data/comt_1_archive/inundation_tropical/UND_ADCIRC/Hurricane_Ike_3D_final_run_with_waves.ascii?sigma[0:1:10]

guygriffiths commented 8 years ago

I've had a further look, and the problem is that the requests I'm making are too large. I think that this will be a general issue and that some adjustment is going to be needed for reading over the network - we haven't done a great deal of testing with large OPeNDAP datasets.

rsignell-usgs commented 8 years ago

Perhaps that explains why this dataset took so long to load? Ideally you would just load the node coordinates (and if necessary the face coordinates) along with the mesh_topology array once, and use it for all the data variables. The only other variable that you need to load is the time coordinate variable, right? All this should take less than 30 seconds or so via OPeNDAP.

guygriffiths commented 8 years ago

Actually we load a subsample of all of the variables to automatically determine the scale range to use. The problem was that I had confused when the reading happened in the Netcdf-Java library, and was inadvertently trying to load the entire variable into memory, which was forbidden by the OPeNDAP server. The long load times were related to this - each time I tried to read a single pixel, a new request was made to the server (because the previous attempt had failed and hence the array had not made it to the cache) which would respond with a 403 error. The area we were subsampling was 100x100 pixels, so that's 10,000 failed requests per variable - it's not surprising that it was slow to load! Anyway, I've got it working at a decent speed now with OPeNDAP UGRID requests, although the code's quite messy. I should hopefully have time to tidy it up this afternoon, and I'll do a bugfix release (v2.1.1) either today or (more likely) tomorrow.

rsignell-usgs commented 8 years ago

Could we specify attributes that ncWMS2 would read to set the initial scale range, to avoid the need to read data if these certain attributes exist? I'm thinking like perhaps setting attributes "colorscalerange" or something for each variable.

guygriffiths commented 8 years ago

It's a possibility, but reading a 100x100 sample of data shouldn't slow things down enough to make that necessary. I haven't timed it but I'm pretty sure this dataset takes under 2 minutes to load now - possibly <1min.

rsignell-usgs commented 8 years ago

Okay, great! @gregpike, we shouldn't be hammering that LSU connection anymore once we get this new version installed.

guygriffiths commented 8 years ago

OK, I was actually getting my datasets mixed up. The OPeNDAP dataset in issue https://github.com/Reading-eScience-Centre/edal-java/issues/44 takes ~30s to load, this one seems to be quite a bit slower, but now loads in <4 minutes. Retrieving a full image in Godiva 3 (e.g. height) took ~30s (the other takes ~5s). Not overly fast, but I guess there's more data in this one or the server is a little slower. Either way, I'm pretty confident that this particular speed issue has been fixed now.

I've built a version from the develop branch and sent you a link. If you could test it out on your end and let me know of any issues I'd be grateful. Assuming it all goes smoothly I'll do a bugfix release tomorrow.

rsignell-usgs commented 8 years ago

I'm out of the office most of the day today, but will try to test tonight.

rsignell-usgs commented 8 years ago

Woo hoo! As you noted, this now takes about 3 minutes to load, but only about 30 seconds or less to draw. Looks great! This is a fantastic development!

2016-03-03_21-27-17

rsignell-usgs commented 8 years ago

@kwilcox, check this out!

guygriffiths commented 8 years ago

Good to hear it's all working. I've incorporated these changes into a bugfix release: https://github.com/Reading-eScience-Centre/edal-java/releases/tag/edal-1.1.1