org.apache.solr.common.SolrException: undefined field ClassHierarchy

[muppsy007]

Silverstripe version: 3.6 Fulltextsearch version: 2.4 or 2.2 Solr version: 6.6.6, 7.7.1, 8.0.0

I've tried multiple versions of Solr with the same result. I can't for the life of me get the following setup and snippet to work:

_config.php

Solr::configure_server(array(
    'host' => '127.0.0.1', // default: localhost | The host or IP Solr is listening on
    'port' => '8983', // default: 8983 | The port Solr is listening on
    'indexstore' => array(
        'mode' => 'file', // a classname which implements SolrConfigStore, or 'file' or 'webdav'
        'path' => BASE_PATH . '/.solr', // The (locally accessible) path to write the index configurations to OR The suburl on the solr host that is set up to accept index configurations via webdav
    )
));

/mysite/code/Search/MyIndex.php

<?php
class MyIndex extends SolrIndex {
    function init() {
        $this->addClass('MyObject');
        $this->addFulltextField('CustomName');
        $this->addFulltextField('CustomDescription');
    }
}

/dev/tasks/Solr_Configure appears to work. (doesn't display anything aside from a title and creates a .solr directory with a subdir 'MyIndex' with a single conf directory containing xml and txt files). I can also visit the Solr admin fine. It's when I run either /dev/tasks/Solr_Reindex that I get a 400 Bad Request on screen, and the following output in log. What am I doing wrong?

org.apache.solr.common.SolrException: undefined field ClassHierarchy
    at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1328)
    at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:455)
    at org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:84)
    at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:191)
    at org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:241)
    at org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:444)
    at org.apache.solr.parser.QueryParser.newFieldQuery(QueryParser.java:61)
    at org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:970)
    at org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:697)
    at org.apache.solr.parser.QueryParser.Term(QueryParser.java:420)
    at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:277)
    at org.apache.solr.parser.QueryParser.Query(QueryParser.java:161)
    at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:281)
    at org.apache.solr.parser.QueryParser.Query(QueryParser.java:161)
    at org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:130)
    at org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:211)
    at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:53)
    at org.apache.solr.search.QParser.getQuery(QParser.java:168)
    at org.apache.solr.update.DirectUpdateHandler2.getQuery(DirectUpdateHandler2.java:429)
    at org.apache.solr.update.DirectUpdateHandler2.deleteByQuery(DirectUpdateHandler2.java:459)
    at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:78)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:59)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalDelete(DistributedUpdateProcessor.java:984)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.versionDeleteByQuery(DistributedUpdateProcessor.java:1675)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.doDeleteByQuery(DistributedUpdateProcessor.java:1580)
    at org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:1389)
    at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processDelete(LogUpdateProcessorFactory.java:124)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:59)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:59)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:59)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:59)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:59)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:59)
    at org.apache.solr.update.processor.FieldNameMutatingUpdateProcessorFactory$1.processDelete(FieldNameMutatingUpdateProcessorFactory.java:79)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:59)
    at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:59)
    at org.apache.solr.handler.loader.XMLLoader.processDelete(XMLLoader.java:366)
    at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:292)
    at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:188)
    at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
    at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
    at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
    at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:724)
    at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:530)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
    at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
    at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
    at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
    at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
    at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
    at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
    at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
    at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
    at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
    at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
    at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
    at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
    at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
    at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
    at org.eclipse.jetty.server.Server.handle(Server.java:534)
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
    at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
    at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
    at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
    at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
    at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
    at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
    at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
    at java.lang.Thread.run(Thread.java:748)

[NightJar]

hi @muppsy007 - thanks for reporting. I was confused at first and this looked like a bug I've seen recently, however on closer look it is complaining about ClassHierarchy not being defined - can you please confirm whether you have a field with this name on any object in your project (including vendor provided portions)?

It seems like perhaps there's a misconfiguration between the generated solr_config (schema.xml iirc), and the reality of the class structure.

[muppsy007]

Hi @NightJar - The only place I find it searching codebase is actually in the fulltextsearchmodule itself. I thought it might have been a standard field from an older version of Silverstripe that was deprecated.

https://github.com/silverstripe/silverstripe-fulltextsearch/blob/b17bee71e5fb09f9eb05e12777158aa616916caf/code/solr/SolrIndex.php#L172

https://github.com/silverstripe/silverstripe-fulltextsearch/blob/b17bee71e5fb09f9eb05e12777158aa616916caf/code/solr/SolrIndex.php#L550

https://github.com/silverstripe/silverstripe-fulltextsearch/blob/b17bee71e5fb09f9eb05e12777158aa616916caf/code/solr/SolrIndex.php#L629

https://github.com/silverstripe/silverstripe-fulltextsearch/blob/b17bee71e5fb09f9eb05e12777158aa616916caf/code/solr/SolrIndex.php#L689

https://github.com/silverstripe/silverstripe-fulltextsearch/blob/b17bee71e5fb09f9eb05e12777158aa616916caf/code/solr/reindex/handlers/SolrReindexBase.php#L213

But this just led me to something. The schema.xml that I is created by the Solr_Configure task. It's defined in there along with my custom index stuff.

<fields>
        <field name='_documentid' type='string' indexed='true' stored='true' required='true' />
        <field name='ID' type='tint' indexed='true' stored='true' required='true' />
        <field name='ClassName' type='string' indexed='true' stored='true' required='true' />
        <field name='ClassHierarchy' type='string' indexed='true' stored='true' required='true' multiValued='true' />
        <field name='_text' type='htmltext' indexed='true' stored='true' multiValued='true' />
        <field name='MyObject_CustomName' type='text' indexed='true' stored='true' multiValued=''/>
        <field name='MyObject_CustomDescription' type='htmltext' indexed='true' stored='true' multiValued=''/>
        <field name='_versionedstage' type='string' indexed='true' stored='true' multiValued=''/>
        <field name="_version_" type="long" indexed="true" stored="true" multiValued="false"/>
    </fields>

I manually removed that field definition from the schema.xml, tried a dev/build, even a solr restart. Same error persists.

[muppsy007]

@NightJar On further tracing I found this is the culprit of my particular error:

https://github.com/silverstripe/silverstripe-fulltextsearch/blob/b17bee71e5fb09f9eb05e12777158aa616916caf/code/solr/SolrIndex.php#L629

// Delete all records which do not match the necessary classname rules
        $conditions = array();
        foreach ($classes as $class => $options) {
            if ($options['include_children']) {
                $conditions[] = "ClassHierarchy:{$class}";
            } else {
                $conditions[] = "ClassName:{$class}";
            }
        }

But it seems like a reliance everywhere on ClassHierarchy that isn't there. Like a mismatch between SS versions? (I can't find any docs on ClassHierarchy). I tried altering my Index config to index standard pages instead of custom objects to rule that out. But I get the same thing.

<?php
class MyIndex extends SolrIndex {
    function init() {
        $this->addClass('Page');
        $this->addFulltextField('Title');
        $this->addFulltextField('Content');
    }
}

[muppsy007]

@NightJar I wonder if Solr just isn't picking up the generated schema.xml and that's what is being referred to as undefined? I just did a simple test search on another field that definately is in the ORM and defined in schema.xml, and I get the same error, but for that field:

$query = new SearchQuery();
$query->search('my query', null, ['MyObject_CustomName' => 1.5, 'MyObject_CustomDescription' => 1.0]);
$results = singleton('MyIndex')->search($query);

org.apache.solr.common.SolrException: undefined field MyObject_CustomName

The config is in docroot/.solr/MyIndex/schema.xml. Interestingly, Solr_Configure task cannot create the Index by itself. Spits out a 400. But if I create the core with the same name manually in Solr dashboard, Solr_Configure runs fine.

[NightJar]

Interesting, thanks for adding this extra info @muppsy007

[muppsy007]

@NightJar This might be it. I found copying the generated conf from the above location to /var/solr/data/MyIndex and reloading the core gets some response from Solr, albeit more errors about unsupported modules and types.

What is the actual recommended version of Solr for the 2.x branch of this module? I don't see anything on the readme, however the standalone solr docs refer to 4.x. In fact, master docs refer to Solr 4.x in places too. Not sure if this is just inherited from old versions of the module, or the module only supports Solr 4.x.

[NightJar]

@muppsy007 to my knowledge the module has only been tested (and thus "supported") on Solr 4 (which unfortunately is quite out of date). However there have been efforts to support more recent versions #226 #120 If I recall correctly, the only major blocker is that there has been limited testing. If you feel that there could be some improvements made to enable more support, please take a look and see if you could help any of the existing efforts :)

Do you think this issue is purely Solr version related?

[muppsy007]

Thanks @NightJar .

I had another go at this last night and managed to get a functioning search system after installing Solr 4.9.1. So I guess the issue here is an incompatibility between the module and more recent versions of Solr.

My functional version combo for future searches:

• Silverstripe 3.6
• fulltextsearch 2.4
• Solr 4.9.1 (latest 4.x)

Thanks for your help.

[NightJar]

Thanks for reporting back @muppsy007 !