sul-dlss / dlme

Digital Library of the Middle East web application, based on Spotlight
https://dlmenetwork.org/
Other
20 stars 2 forks source link

Adding items not working - logging SOLR errors #603

Closed aaron-collier closed 5 years ago

aaron-collier commented 5 years ago

Will add more information as available, for now, the following SOLR error is logged when attempting to ingest an item:

solr_1        | 2019-10-29 20:24:27.341 INFO  (qtp1544078442-21) [   x:dlme] o.a.s.u.p.LogUpdateProcessorFactory [dlme]  webapp=/solr path=/update params={commitWithin=500&wt=json}{} 0 0
solr_1        | 2019-10-29 20:24:27.341 ERROR (qtp1544078442-21) [   x:dlme] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Unknown operation for the an atomic update, operation ignored: none for id:stanford_kj751hs0595
solr_1        |     at org.apache.solr.update.processor.AtomicUpdateDocumentMerger.merge(AtomicUpdateDocumentMerger.java:133)
solr_1        |     at org.apache.solr.update.processor.DistributedUpdateProcessor.getUpdatedDocument(DistributedUpdateProcessor.java:694)
solr_1        |     at org.apache.solr.update.processor.DistributedUpdateProcessor.doVersionAdd(DistributedUpdateProcessor.java:372)
solr_1        |     at org.apache.solr.update.processor.DistributedUpdateProcessor.lambda$versionAdd$0(DistributedUpdateProcessor.java:337)
solr_1        |     at org.apache.solr.update.VersionBucket.runWithLock(VersionBucket.java:50)
solr_1        |     at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:337)
solr_1        |     at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:223)
solr_1        |     at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
solr_1        |     at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:507)
solr_1        |     at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:145)
solr_1        |     at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:121)
solr_1        |     at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:84)
solr_1        |     at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
solr_1        |     at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
solr_1        |     at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
solr_1        |     at org.apache.solr.core.SolrCore.execute(SolrCore.java:2578)
solr_1        |     at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:780)
solr_1        |     at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
solr_1        |     at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:423)
solr_1        |     at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:350)
solr_1        |     at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
solr_1        |     at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
solr_1        |     at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
solr_1        |     at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
solr_1        |     at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
solr_1        |     at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
solr_1        |     at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1711)
solr_1        |     at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
solr_1        |     at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1347)
solr_1        |     at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
solr_1        |     at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
solr_1        |     at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1678)
solr_1        |     at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
solr_1        |     at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1249)
solr_1        |     at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
solr_1        |     at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:220)
solr_1        |     at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:152)
solr_1        |     at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
solr_1        |     at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
solr_1        |     at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
solr_1        |     at org.eclipse.jetty.server.Server.handle(Server.java:505)
solr_1        |     at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
solr_1        |     at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
solr_1        |     at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
solr_1        |     at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
solr_1        |     at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
solr_1        |     at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:781)
solr_1        |     at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:917)
solr_1        |     at java.base/java.lang.Thread.run(Thread.java:834)
solr_1        |
aaron-collier commented 5 years ago

I think it's likely that we've drifted from our solr config: https://github.com/sul-dlss/dlme/blob/master/solr/config/solrconfig.xml

ndushay commented 5 years ago

more likely the Solr schema is the issue, but same general gist, yeah.

Not sure if this is useful: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201606.mbox/%3Ccfe3d868-1911-7c12-8d6f-73d7bd60c638@elyograg.org%3E

This happens when you add a document where the value of a field is a
key/value construct, but isn't actually an atomic update.  In JSON, this
is represented with curly braces.  In Java, it is a Map object.  This
kind of construct is only used for Atomic Updates -- the key must be one
of the atomic operations: set, add, delete, inc.  The warning message
that I quoted indicates that the the key in the key/value construct for
that document was 3932885930.

If you are using SolrJ, then the value of one or more fields is being
set to a Map object, which isn't right unless you intend to do an Atomic
Update.  If you are using JSON formatted updates, there are probably
curly braces where they don't belong.

esp If you are using JSON formatted updates, there are probably curly braces where they don't belong.

mjgiarlo commented 5 years ago

The "bug" was when we would run multiple traject configs, each using convert_to_language_hash on the same field.