search

AtlasOfLivingAustralia / layers-service

Spatial layers - this repo for issues/doc only, not code
3 stars 4 forks source link

Load layers : Fix MDBA indigenous language boundaries layers #105

Closed ansell closed 5 years ago

ansell commented 5 years ago

Fixes have been published for the four MDBA indigenous language boundaries layers that were loaded in 2016.

https://data.gov.au/dataset/murray-darling-basin-water-resource-plan-areas-surface-water https://data.gov.au/dataset/murray-darling-basin-water-resource-plan-areas-groundwater https://data.gov.au/dataset/surface-water-sdl-resource-units https://data.gov.au/dataset/groundwater-sdl-resource-units https://data.gov.au/dataset/a-guide-to-traditional-owner-groups-for-water-resource-plan-areas

Refs ticket numbers 21350 24646 and 26861 in helpdesk

ansell commented 5 years ago

The relevant layer/field ids that need modifying as part of this are:

layer id
mdb_surface_water_water_resource_plan_areas 10834
mdb_ground_water_water_resource_plan_areas 10835
mdb_surface_water_sdl_resource_units 10832
mdb_ground_water_sdl_resource_units 10833
gw_wrpa_traditional_owners 10848
sw_wrpa_traditional_owners 10847
ansell commented 5 years ago

Some of the layers above may not have been modified. Comparing dates in metadata is the guidance I have received on identifying which of the layers above need to actually be reloaded.

adam-collins commented 5 years ago

Current versions of these layers are loaded on staging.

new layer new id
sw_areas 10905
gw_areas 10906
sw_sdl 10907
gw_sdl 10908
gw_trad 10909
sw_trad 10910
ansell commented 5 years ago

@adam-collins

I checked that the fields are now available in Solr, which seems to be successful:

https://mdba.ala.org.au/occurrences/b63c148c-077e-4d32-9e58-9f34f86e598f

https://biocache-ws.ala.org.au/ws/occurrences/b63c148c-077e-4d32-9e58-9f34f86e598f

However, updating the layers in the grouped_facets.json file caused a grails error when attempting to visit the search results page https://mdba.ala.org.au/occurrences/search?fq=data_hub_uid%3Adh10&q=

The log files show the following:

==> /var/log/tomcat7/catalina.out <==
2019-01-23 12:01:08,348 ERROR [GrailsExceptionResolver]  NullPointerException occurred when processing request: [GET] /occurrences/search
Cannot invoke method replaceAll() on null object. Stacktrace follows:
org.codehaus.groovy.grails.web.pages.exceptions.GroovyPagesException: Error processing GroovyPageView: Cannot invoke method replaceAll() on null object
    at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:198)
    at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)
    at au.org.ala.cas.client.AlaHttpServletRequestWrapperFilter.doFilter(AlaHttpServletRequestWrapperFilter.java:66)
    at au.org.ala.cas.client.UriFilter.doFilter(UriFilter.java:197)
    at org.jasig.cas.client.validation.AbstractTicketValidationFilter.doFilter(AbstractTicketValidationFilter.java:196)
    at au.org.ala.cas.client.UriFilter.doFilter(UriFilter.java:197)
    at org.jasig.cas.client.authentication.AuthenticationFilter.doFilter(AuthenticationFilter.java:93)
    at au.org.ala.cas.client.UriFilter.doFilter(UriFilter.java:197)
    at org.jasig.cas.client.session.SingleSignOutFilter.doFilter(SingleSignOutFilter.java:65)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: Cannot invoke method replaceAll() on null object
    at gsp_mdbaHub_occurrencelist_gsp$_run_closure1.doCall(gsp_mdbaHub_occurrencelist_gsp.groovy:58)
    at gsp_mdbaHub_occurrencelist_gsp.run(gsp_mdbaHub_occurrencelist_gsp.groovy:126)
    ... 12 more

==> /var/log/nginx/access.log <==
[REDACTED] - - [23/Jan/2019:12:01:08 +1100] "GET /occurrences/search?fq=data_hub_uid%3Adh10&q= HTTP/2.0" 500 9171 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:64.0) Gecko/20100101 Firefox/64.0" "-"

This was the change that was made to the facets file (on a branch because of how it breaks mdba):

https://github.com/AtlasOfLivingAustralia/ala-install/commit/df615d8aef0ef8ab23311cad59289f7ca3c8f2f9

I have left the updated facets json file deployed to mdba-test.ala.org.au so the error on this page should be reproducible until the next time it is deployed or a remaining underlying layer issue is fixed:

https://mdba-test.ala.org.au/occurrences/search?fq=data_hub_uid%3Adh10&q=

ansell commented 5 years ago

https://biocache-ws-test.ala.org.au/ws/occurrences/b63c148c-077e-4d32-9e58-9f34f86e598f (for example) shows cl10906 as having a value, but that may be coming from Cassandra rather than Solr.

The biocache-service query being made by mdba-hub is failing with an error from solr about the cl10906 field being undefined, as seen below on test-bws-b5-2:

==> /var/log/tomcat7/biocache-service.log <==
2019-01-23 13:07:19,532 [http-bio-8080-exec-8] ERROR org.apache.solr.client.solrj.impl.CloudSolrClient  (CloudSolrClient.java:1167) - Request to collection biocache failed due to (400) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://aws-sc3b.ala:8983/solr/biocache: Error from server at http://aws-sc4b.ala:8983/solr/biocache-2019-01-23-06-49_shard3_replica1: undefined field: "cl10906", retry? 0
2019-01-23 13:07:19,532 [http-bio-8080-exec-8] ERROR au.org.ala.biocache.dao.SearchDAOImpl  (SearchDAOImpl.java:3978) - query failed: qt=standard&facet=true&facet.field=cl3004&facet.field=cl3003&facet.field=cl2113&facet.field=cl10906&facet.field=cl10907&facet.field=cl10908&facet.field=cl10905&facet.field=cl10829&facet.field=cl10831&facet.field=species_group&facet.field=common_name&facet.field=species&facet.field=subspecies_name&facet.field=family&facet.field=life_stage&facet.field=elevation_d_rng&facet.field=raw_sex&facet.field=state&facet.field=cl959&facet.field=coordinate_uncertainty&facet.field=state_conservation&facet.field=cl1918&facet.field=cl617&facet.field=cl620&facet.field=cl1048&facet.field=cl21&facet.field=month&facet.field=occurrence_decade_i&facet.field=basis_of_record&facet.field=multimedia&facet.field=data_provider_uid&facet.field=data_resource_uid&f.species_group.facet.sort=index&f.common_name.facet.sort=index&f.species.facet.sort=index&f.subspecies_name.facet.sort=index&f.family.facet.sort=index&f.life_stage.facet.sort=count&f.raw_sex.facet.sort=count&f.state.facet.sort=index&f.cl959.facet.sort=count&f.coordinate_uncertainty.facet.sort=index&f.state_conservation.facet.sort=count&f.cl1918.facet.sort=count&f.cl617.facet.sort=count&f.cl620.facet.sort=count&f.cl1048.facet.sort=count&f.cl21.facet.sort=count&f.month.facet.sort=index&f.occurrence_decade_i.facet.sort=index&f.basis_of_record.facet.sort=index&f.multimedia.facet.sort=count&f.data_provider_uid.facet.sort=count&f.data_resource_uid.facet.sort=count&facet.mincount=1&facet.limit=10&facet.sort=count&rows=20&start=0&facet.range.other=before&facet.range.other=after&q=*:*&fq=data_hub_uid:dh10&fq=cl2110:"Murray-Darling+Basin+Boundary"&facet.missing=true&sort=first_loaded_date+desc&_stateVer_=biocache-2019-01-23-06-49:35 : Error from server at http://aws-sc3b.ala:8983/solr/biocache: Error from server at http://aws-sc4b.ala:8983/solr/biocache-2019-01-23-06-49_shard3_replica1: undefined field: "cl10906"
2019-01-23 13:07:19,533 [http-bio-8080-exec-8] ERROR au.org.ala.biocache.dao.SearchDAOImpl  (SearchDAOImpl.java:689) - Error executing query with requestParams: q=*:*&fq=data_hub_uid:dh10&start=0&pageSize=20&sort=first_loaded_date&dir=desc&qc=cl2110:"Murray-Darling Basin Boundary"&facets=cl3004&facets=cl3003&facets=cl2113&facets=cl10906&facets=cl10907&facets=cl10908&facets=cl10905&facets=cl10829&facets=cl10831&facets=species_group&facets=common_name&facets=species&facets=subspecies_name&facets=family&facets=life_stage&facets=elevation_d_rng&facets=raw_sex&facets=state&facets=cl959&facets=coordinate_uncertainty&facets=state_conservation&facets=cl1918&facets=cl617&facets=cl620&facets=cl1048&facets=cl21&facets=month&facets=occurrence_decade_i&facets=basis_of_record&facets=multimedia&facets=data_provider_uid&facets=data_resource_uid&flimit=10&formattedQuery=*:*&facet=true
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://aws-sc3b.ala:8983/solr/biocache: Error from server at http://aws-sc4b.ala:8983/solr/biocache-2019-01-23-06-49_shard3_replica1: undefined field: "cl10906"
    at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:612)
    at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
    at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
    at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
    at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
    at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
    at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
    at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
    at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
    at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)
    at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:990)
    at au.org.ala.biocache.dao.SearchDAOImpl.query(SearchDAOImpl.java:3959)
    at au.org.ala.biocache.dao.SearchDAOImpl.runSolrQuery(SearchDAOImpl.java:2683)
    at au.org.ala.biocache.dao.SearchDAOImpl.findByFulltextSpatialQuery(SearchDAOImpl.java:675)
    at au.org.ala.biocache.dao.SearchDAOImpl.findByFulltextSpatialQuery(SearchDAOImpl.java:653)
    at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
    at com.sun.proxy.$Proxy44.findByFulltextSpatialQuery(Unknown Source)
    at au.org.ala.biocache.web.OccurrenceController.occurrenceSearch(OccurrenceController.java:819)
    at sun.reflect.GeneratedMethodAccessor267.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:743)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:672)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:82)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:933)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:867)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:951)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:842)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:827)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at au.org.ala.biocache.web.SimpleCORSFilter.doFilter(SimpleCORSFilter.java:33)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at au.org.ala.biocache.web.LoggerConfigFilter.doFilter(LoggerConfigFilter.java:26)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:129)
    at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:77)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:106)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:221)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

==> /var/log/nginx/access.log <==
[REDACTED] - - [23/Jan/2019:13:07:19 +1100] "GET /ws/occurrences/search?q=*%3A*&fq=data_hub_uid%3Adh10&start=0&pageSize=20&sort=first_loaded_date&dir=desc&qc=cl2110%3A%22Murray-Darling%20Basin%20Boundary%22&facets=cl3004&facets=cl3003&facets=cl2113&facets=cl10906&facets=cl10907&facets=cl10908&facets=cl10905&facets=cl10829&facets=cl10831&facets=species_group&facets=common_name&facets=species&facets=subspecies_name&facets=family&facets=life_stage&facets=elevation_d_rng&facets=raw_sex&facets=state&facets=cl959&facets=coordinate_uncertainty&facets=state_conservation&facets=cl1918&facets=cl617&facets=cl620&facets=cl1048&facets=cl21&facets=month&facets=occurrence_decade_i&facets=basis_of_record&facets=multimedia&facets=data_provider_uid&facets=data_resource_uid&facets=institution_uid&facets=collection_uid&facets=alau_user_id&facets=outlier_layer&facets=outlier_layer_count&facets=duplicate_status&facets=duplicate_type&flimit=10 HTTP/1.1" 200 214 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:64.0) Gecko/20100101 Firefox/64.0" "[REDACTED]" request_time=0.139 upstream_response_time=0.139 upstream_connect_time=0.001 upstream_header_time=0.139 upstream_cache_status=-

It isn't clear to me why Grails is falling over on a replaceAll rather than showing an error when the query fails in this way, but it looks like a red herring in this case.

ansell commented 5 years ago

New index with these fields has been generated and the current record counts for validation purposes are as follows:

new layer new id record count
sw_areas 10905 18824797
gw_areas 10906 18824856
sw_sdl 10907 18789783
gw_sdl 10908 18515747
gw_trad 10909 18824856
sw_trad 10910 18824797