search

mskyttner / clb-docker

GBIF taxonomy tools dockerized
GNU Affero General Public License v3.0
3 stars 3 forks source link

HDFS-related error message #6

Open mskyttner opened 7 years ago

mskyttner commented 7 years ago

Attempting a seach for "Vulpes", using http://clb.docker/species/search?q=Vulpes, gives a Error 500 with the following information indicating an issue with solr config:

org.apache.solr.common.SolrException: SolrCore 'checklistbank' is not available due to init failure: Unrecognized lockType: hdfs at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:997) at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:147) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)

mdoering commented 7 years ago

The hdfs locktype only works in solr cloud which we use at GBIF. It should have been removed from the solrconfig.xml in this docker step: https://github.com/mskyttner/clb-docker/blob/master/solr/Dockerfile#L37

mskyttner commented 7 years ago

@mdoering that step seems execute in the branch #9, yet we still get an issue when accessing /species/search, and it should be possible to replicate this issue using that branch using "firefox http://clb.docker/species/search". We now get:

org.apache.solr.common.SolrException: SolrCore 'checklistbank' is not available due to init failure: Unrecognized lockType: hdfs

We tried to modify the solrconfig.xml file by removing these lines: https://github.com/gbif/checklistbank/blob/master/checklistbank-solr/src/main/resources/solr/checklistbank/conf/solrconfig.xml#L6-L10

This removed a warning for the solr logs but did not solve the issue. Any advice?

mdoering commented 7 years ago

The problematic line to be removed is https://github.com/gbif/checklistbank/blob/master/checklistbank-solr/src/main/resources/solr/checklistbank/conf/solrconfig.xml#L34

mskyttner commented 7 years ago

Yes, and that seems to have worked, we verified that the sed command removes it in this step (https://github.com/mskyttner/clb-docker/blob/master/solr/Dockerfile#L37) using this command:

root@solr:/opt/solr# grep hdfs server/solr/checklistbank/conf/solrconfig.xml

But we still get that error message....

mskyttner commented 7 years ago

@mdoering - the solrconfig.xml has no "hdfs" string in it - something else we should check?

This works:

firefox http://clb.docker/species?name=Vulpes%20vulpes

Not this:

firefox http://clb.docker/species/search?q=Vulpes+vulpes

mdoering commented 7 years ago

The first URL is a call to the db only, the 2nd uses solr. Something hosed but no idea what that is without properly debugging the installation. Trying to find some time to do that soon

mskyttner commented 7 years ago

@mdoering Thanks for having a look! For what it is worth, this command gives nothing in the solr container: root@solr:/opt/solr/server/solr# grep -r hdfs * so it doesn't seem to be anything in text config files there? Could it be something inside a jar?

mdoering commented 7 years ago

could you check the file from within the solr UI? that shows the actual config in use. maybe it gets it from somewhere else...

mskyttner commented 7 years ago

I couldn't find anything in the solr UI, I did a Ctrl+F on all the tabs I could find in the UI on "hdfs". The stack trace when I get http://clb.docker/species/search?q=Vulpes+vulpes says:

org.apache.solr.common.SolrException: Unrecognized lockType: hdfs
    at org.apache.solr.core.StandardDirectoryFactory.createLockFactory(StandardDirectoryFactory.java:74)
    at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:349)
    at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:301)
    at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:516)

Could it be some default setting or something kicking in then?