search

aodn / aodn-portal

AODN Open Geospatial Portal
https://portal.aodn.org.au/
GNU General Public License v3.0
21 stars 13 forks source link

wfs scanner creates a heavy load on geoserver #1101

Closed jonescc closed 10 years ago

jonescc commented 10 years ago

While investigating the 'sluggishness' of geoserver-rc issue, I notice that the wfs scanner is taking up a number of connections in the database and slowing down real downloads.

Would be good if we could throttle the scanner so that it only uses one connection at most.

See an extract of wfs scanner log entries from geoserver_rc apache access log below. The wfs scanner is effectively trying to download all layer content from the database every couple of hours and its doing so using a number of connections (if it was one after another the times would be sequential), exacerbating other connection issues.

03.6.253.88 - - [02/May/2014:14:27:11 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:csiro_harvest_nrs_zooplankton_map HTTP/1.1" 200 114873 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:11 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:csiro_harvest_nrs_biomass_map HTTP/1.1" 200 114681 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:15 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:ctd_sattag_nz_fur_seal_vw HTTP/1.1" 200 124665 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:16 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:BioOptical_deployments HTTP/1.1" 200 134649 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:16 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:anfog_realtime HTTP/1.1" 200 115673 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:17 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:srs_oc_soop_rad HTTP/1.1" 200 119692 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:21 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:soop_sst_1min_vw HTTP/1.1" 200 120359 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:38 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:so_cpr_geoserver_view HTTP/1.1" 200 114662 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:32 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=aodn:south_australian_coastal_data HTTP/1.1" 200 114662 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:28:01 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:soop_tmv HTTP/1.1" 200 116436 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:28:09 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:soop_trv_mv HTTP/1.1" 200 119886 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:32 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:ctd_sattag_southern_elephant_seal_vw HTTP/1.1" 200 116646 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:26 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:soop_sst_mv HTTP/1.1" 200 121742 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:31 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:soop_sst_recent_vw HTTP/1.1" 200 150702 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:30:35 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=aodn:water_levels_gippsland_lakes HTTP/1.1" 200 29336 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:30:35 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=aodn:water_sampling_cockburn_sound HTTP/1.1" 200 32770 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:30:35 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=aodn:water_sampling_cockburn_sound_1982-2012 HTTP/1.1" 200 26469 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:30:46 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:aatams_biologging_shearwater_map HTTP/1.1" 200 81870 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:27:11 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:csiro_harvest_nrs_biomass_data HTTP/1.1" 200 107654 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:30:33 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:soop_co2 HTTP/1.1" 200 121742 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:30:35 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=aodn:water_sampling_port_jackson HTTP/1.1" 200 115673 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:30:36 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:ctd_sattag_weddell_seal_vw HTTP/1.1" 200 118721 "-" "Java/1.7.0_51" 103.6.253.88 - - [02/May/2014:14:30:37 +1000] "GET /geoserver/wms?SERVICE=WFS&VERSION=1.1.1&REQUEST=GetFeature&typeName=imos:soop_xbt HTTP/1.1" 200 115809 "-" "Java/1.7.0_51" 10

jonescc commented 10 years ago

Looks like the wfs scanner is creating a connection to geoserver, aborting because the cache directory is not configured correctly, not closing the connection and going onto the next request and repeating, meaning a number of connections are being opened to geoserver at the same time.

Needs a fix to chef see above to configure cache directory correctly and to the wfsscanner to close connections if there is an exception

jonescc commented 10 years ago

Main issue was cache directory not being set by chef causing each request to fail and go onto the next not closing the http connection to tomcat. Have fixed cache directory issue in chef and modified the wfs scanner to close connections/reuse existing connections where it can. Have also modified the scanner to run once a day as it is now taking around 6 hours to complete a scan.