search

codelibs / fess

Fess is very powerful and easily deployable Enterprise Search Server.
https://fess.codelibs.org
Apache License 2.0
993 stars 166 forks source link

BUG! exception in phase... with standalone Elasticsearch #2698

Closed rkfyi closed 1 year ago

rkfyi commented 1 year ago

I followed the installation instructions for Fess and Elasticsearch 8.5.1 on Windows. That means I downloaded the zip files, extracted them appropriately, and modified the files as instructed. For Elasticsearch no textual modification was needed. One just launches elasticsearch.bat. Notably, no supervisor credentials appeared during the setup process, even though the installation reference document said they would. Here is the result of opening localhost:9200 in Firefox:

{ "name" : "HIDDEN", "cluster_name" : "elasticsearch", "cluster_uuid" : "HIDDENFORTHISFORUMPOST", "version" : { "number" : "8.5.1", "build_flavor" : "default", "build_type" : "zip", "build_hash" : "HIDDENFORTHISFORUMPOST", "build_date" : "HIDDENFORTHISFORUMPOST", "build_snapshot" : false, "lucene_version" : "9.4.1", "minimum_wire_compatibility_version" : "7.17.0", "minimum_index_compatibility_version" : "7.0.0" }, "tagline" : "You Know, for Search" } I followed the fess instructions for making the configsync folder, modifying the YML documents, running the plugin command and so forth. Fess recognizes the elasticsearch instance and removes the development mode warning.

All of the crawl jobs fail. I tried an index job and it also failed. The error message is BUG! exception in phase 'class generation' in source unit 'Script1.groovy' unexpected NullPointerException

I modified the job instruction in the scheduler per error crawling · Issue #2147 · codelibs/fess · GitHub.

return container.getComponent("crawlJob").logLevel("info").gcLogging().execute(executor); return container.getComponent("crawlJob").logLevel("info"); return container.getComponent("crawlJob").logLevel("info").execute(executor); return container.getComponent("crawlJob").execute(executor); All fail. It seems that no crawl jobs work.

Fess interacts with the Elasticsearch instance and this is seen via output such as [2022-12-09T13:32:43,845][INFO ][o.e.c.r.a.AllocationService] [REDACTED] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[fess.suggest.20221209133243][0]]])." previous.health="YELLOW" reason="shards started [[fess.suggest.20221209133243][0]]" on the elasticsearch console

marevol commented 1 year ago

What is the version of Fess and Java?

rkfyi commented 1 year ago

Fess is using Adoptium 17.0.5.8 and ElasticSearch is using the built-in Oracle 19.0.1

From: Shinsuke Sugaya @.> Sent: Friday, December 9, 2022 5:51 PM To: codelibs/fess @.> Cc: rkfyi @.>; Author @.> Subject: Re: [codelibs/fess] BUG! exception in phase... with standalone Elasticsearch (Issue #2698)

What is the version of Fess and Java?

— Reply to this email directly, view it on GitHub https://github.com/codelibs/fess/issues/2698#issuecomment-1344895258 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AXQ362Z4MTILJTBON7T4RWTWMPAXDANCNFSM6AAAAAASZXLCC4 . You are receiving this because you authored the thread.Message ID: @.***>

marevol commented 1 year ago

Could you provide the stack trace in fess.log?

rkfyi commented 1 year ago

fess.log The log is attached. Thank you!

This is the general trace for each of the various jobs. It produces this for every job in the log, whether thumbnail generation, or something else.

BUG! exception in phase 'class generation' in source unit 'Script1.groovy' unexpected NullPointerException
    at org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:942) ~[fess-script-groovy-14.5.0.jar:?]
    at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:692) ~[fess-script-groovy-14.5.0.jar:?]
    at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:666) ~[fess-script-groovy-14.5.0.jar:?]
    at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:373) ~[fess-script-groovy-14.5.0.jar:?]
    at groovy.lang.GroovyClassLoader.lambda$parseClass$2(GroovyClassLoader.java:316) ~[fess-script-groovy-14.5.0.jar:?]
    at org.codehaus.groovy.runtime.memoize.StampedCommonCache.compute(StampedCommonCache.java:163) ~[fess-script-groovy-14.5.0.jar:?]
    at org.codehaus.groovy.runtime.memoize.StampedCommonCache.getAndPut(StampedCommonCache.java:154) ~[fess-script-groovy-14.5.0.jar:?]
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:314) ~[fess-script-groovy-14.5.0.jar:?]
    at groovy.lang.GroovyShell.parseClass(GroovyShell.java:572) ~[fess-script-groovy-14.5.0.jar:?]
    at groovy.lang.GroovyShell.parse(GroovyShell.java:585) ~[fess-script-groovy-14.5.0.jar:?]
    at groovy.lang.GroovyShell.parse(GroovyShell.java:597) ~[fess-script-groovy-14.5.0.jar:?]
    at groovy.lang.GroovyShell.evaluate(GroovyShell.java:459) ~[fess-script-groovy-14.5.0.jar:?]
    at groovy.lang.GroovyShell.evaluate(GroovyShell.java:495) ~[fess-script-groovy-14.5.0.jar:?]
    at groovy.lang.GroovyShell.evaluate(GroovyShell.java:469) ~[fess-script-groovy-14.5.0.jar:?]
    at org.codelibs.fess.script.groovy.GroovyEngine.evaluate(GroovyEngine.java:40) ~[fess-script-groovy-14.5.0.jar:?]
    at org.codelibs.fess.job.impl.ScriptExecutor.execute(ScriptExecutor.java:31) ~[classes/:?]
    at org.codelibs.fess.app.job.ScriptExecutorJob.process(ScriptExecutorJob.java:101) ~[classes/:?]
    at org.codelibs.fess.app.job.ScriptExecutorJob.run(ScriptExecutorJob.java:42) ~[classes/:?]
    at org.lastaflute.job.LaJobRunner.actuallyRun(LaJobRunner.java:267) ~[lasta-job-0.5.6.jar:?]
    at org.lastaflute.job.LaJobRunner.doRun(LaJobRunner.java:234) ~[lasta-job-0.5.6.jar:?]
    at org.lastaflute.job.LaJobRunner.run(LaJobRunner.java:184) ~[lasta-job-0.5.6.jar:?]
    at org.lastaflute.job.cron4j.Cron4jTask.runJob(Cron4jTask.java:432) ~[lasta-job-0.5.6.jar:?]
    at org.lastaflute.job.cron4j.Cron4jTask.actuallyExecute(Cron4jTask.java:406) ~[lasta-job-0.5.6.jar:?]
    at org.lastaflute.job.cron4j.Cron4jTask.lambda$doExecute$6(Cron4jTask.java:309) ~[lasta-job-0.5.6.jar:?]
    at org.lastaflute.job.cron4j.Cron4jTask.synchronizedNeighborRunning(Cron4jTask.java:379) ~[lasta-job-0.5.6.jar:?]
    at org.lastaflute.job.cron4j.Cron4jTask.doExecute(Cron4jTask.java:297) ~[lasta-job-0.5.6.jar:?]
    at org.lastaflute.job.cron4j.Cron4jTask.execute(Cron4jTask.java:199) ~[lasta-job-0.5.6.jar:?]
    at it.sauronsoftware.cron4j.RomanticCron4jNativeTaskExecutor$RomanticRunner.run(RomanticCron4jNativeTaskExecutor.java:121) ~[lasta-job-0.5.6.jar:?]
    at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: java.lang.NullPointerException](url)
rkfyi commented 1 year ago

I switched to using the same Java as Fess using. This is from the elasticsearch.log. Still researching potential solutions. I think some documentation on setting up shards or something might help. It seems like the Elasticsearch installation instructions skip some steps that Fess needs.

[2022-12-17T21:11:00,419][WARN ][r.suppressed             ] [6570B] path: /fess_config.scheduled_job/_search, params: {typed_keys=true, max_concurrent_shard_requests=5, index=fess_config.scheduled_job, batched_reduce_size=512, ccs_minimize_roundtrips=true}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
    at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:728) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:418) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:760) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:512) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.AbstractSearchAsyncAction.performPhaseOnShard(AbstractSearchAsyncAction.java:326) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.AbstractSearchAsyncAction.run(AbstractSearchAsyncAction.java:262) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:469) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:217) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.TransportSearchAction.executeSearch(TransportSearchAction.java:1041) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.TransportSearchAction.executeLocalSearch(TransportSearchAction.java:769) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.TransportSearchAction.lambda$executeRequest$6(TransportSearchAction.java:390) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:162) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:112) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:77) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.TransportSearchAction.executeRequest(TransportSearchAction.java:478) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:277) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:103) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:86) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:53) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:84) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.tasks.TaskManager.registerAndExecute(TaskManager.java:202) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.client.internal.node.NodeClient.executeLocally(NodeClient.java:112) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.rest.action.RestCancellableNodeClient.doExecute(RestCancellableNodeClient.java:81) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:380) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.rest.action.search.RestSearchAction.lambda$prepareRequest$2(RestSearchAction.java:120) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:103) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.xpack.security.rest.SecurityRestFilter.handleRequest(SecurityRestFilter.java:119) ~[?:?]
    at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:396) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:530) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:311) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:379) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:460) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:353) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.handlePipelinedRequest(Netty4HttpPipeliningHandler.java:128) ~[?:?]
    at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:118) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
    at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327) ~[?:?]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:299) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
    at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[?:?]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[?:?]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[?:?]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?]
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[?:?]
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722) ~[?:?]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:623) ~[?:?]
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:586) ~[?:?]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) ~[?:?]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995) ~[?:?]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
    at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: org.elasticsearch.action.NoShardAvailableActionException
    at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:547) ~[elasticsearch-8.5.1.jar:?]
    at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:494) ~[elasticsearch-8.5.1.jar:?]
    ... 73 more
rkfyi commented 1 year ago

I filled out details in the elasticsearch.yml. Previously had not done that. Now crawler works, index works. I know this because the Job log shows that they complete okay. But no search results are returned when processing a query. I type things that I know are on the single website searched. The dashboard says 289 docs and 1.44 MB data. I am testing with a single website. I have not modified any of the jsp files or Fess configuration other than the elastic search connections on fess.in.bat.

rkfyi commented 1 year ago

Suggest Indexer Ok 2022-12-17T21:44:01 2022-12-17T21:46:21 Default Crawler Ok 2022-12-17T21:36:42 2022-12-17T21:42:17

But no results in the search results and I am typing words that I know are on the page and that work in the standalone development mode installation.

marevol commented 1 year ago

Our test team checked on the following environment. It works, and the problem is not reproduced.