populate the webapp with initial configuration and a message before initial index is done

[YoAm]

Seeing the following after starting the container:

There was an error!

CONFIGURATION parameter has not been configured in web.xml! Please configure your webapp.

Unable to determine source root path. Missing configuration?

java.io.FileNotFoundException: Unable to determine source root path. Missing configuration? at org.opensolaris.opengrok.web.PageConfig.checkSourceRootExistence(PageConfig.java:1494) at org.apache.jsp.index_jsp._jspService(index_jsp.java:222) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:444) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:385) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:329) at javax.servlet.http.HttpServlet.service(HttpServlet.java:741) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.opensolaris.opengrok.web.StatisticsFilter.doFilter(StatisticsFilter.java:55) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.opensolaris.opengrok.web.AuthorizationFilter.doFilter(AuthorizationFilter.java:83) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:475) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:80) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:651) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:498) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:796) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1374) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)

[tarzanek]

did you properly mount the source root to /src ? I played with the image too (from opengrok/docker:latest ) and I think I had to do some checks in the image (and ev. restart tomcat to pickup the data dir once indexer is done ... so this could be improved to avoid above error)

[YoAm]

I opened bash on the container and verified the source directory is correctly mounted in /src.

2017-11-13 11:53 GMT+02:00 Lubos Kosco notifications@github.com:

did you properly mount the source root to /src ? I played with the image too (from opengrok/docker:latest ) and I think I had to do some checks in the image (and ev. restart tomcat to pickup the data dir once indexer is done ... so this could be improved to avoid above error)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/OpenGrok/docker/issues/6#issuecomment-343867653, or mute the thread https://github.com/notifications/unsubscribe-auth/AY-_Wl1yQdMuR5VJBh8aGc79i-Ycx5axks5s2BGugaJpZM4QbX9w .

[tarzanek]

are you able to check if indexing was done properly (/var/opengrok/log/... ) and restart the tomcat container ? (and this needs to be fixed)

[YoAm]

Could you please provide more detailed instructions? Is there a string or pattern that signifies indexing was done properly? How do I restart the tomcat container - is there a command I need to run? Do I run it within the opengrok container or from the docker host?

[YoAm]

I've restarted tomcat using the shutdown and startup scripts in /usr/local/tomcat, and the error still shows.

[tarzanek]

/var/opengrok/log/opengrok0.0.log usually says if indexing is done or points to any errors ...

I will try to retest and fix what is needed, anyways we have a repo in official repo in opengrok/docker so plain: docker run -d -v /path/to/mysources/:/src -p 8080:8080 opengrok/docker:latest worked for me ...

[tarzanek]

(and if not I did: bash -c "clear && docker exec -it docker_machine_name bash" and did the tomcat restart using /etc/init.d I think )

[YoAm]

greping for "index", all I see are these log prints:

/var/opengrok/log/opengrok0.0.log:2017-11-14 12:25:06.273+0000 INFO t1 Indexer.prepareIndexer: Scanning for repositories... /var/opengrok/log/opengrok0.0.log:2017-11-14 12:25:14.037+0000 INFO t1 Indexer.prepareIndexer: Done scanning for repositories (7s)

There are no errors. I do see these warnings but I don't think they're critical:

/var/opengrok/log/opengrok0.4.log:2017-11-13 22:06:30.969+0000 WARNING t24 Executor.exec: Non-zero exit status 137 from command [/usr/bin/git, log, --abbrev-commit, --abbrev=8, --name-only, --pretty=fuller, --date=iso8601-strict, --follow, --, ] in directory /src/5.5 /var/opengrok/log/opengrok0.4.log:2017-11-13 22:06:31.076+0000 WARNING t24 FileHistoryCache$1.run: doFileHistory() got exception

[tarzanek]

did it say "sending config to XYZ" somewhere ? or something like one indexing ?

[YoAm]

grep for "sending config" and "one index" didn't return anything. I noticed that git was using up ~25% CPU so I guess that's the cron indexing job.

[tarzanek]

git is just one part of the story can you verify there is content in /var/opengrok/data and empty timestamp file in /var/opengrok/data/ ? also that error means that basically it is looking for /var/opengrok/etc/configuration.xml if it is present ...

[YoAm]

Is the timestamp file meant to be named "timestamp"?

root@46a6526c1312:/usr/local/tomcat# ls -la /var/opengrok/data/ total 20 drwxr-xr-x 5 root root 4096 Nov 13 19:57 . drwxr-xr-x 1 root root 4096 Nov 13 08:01 .. drwxr-xr-x 3 root root 4096 Nov 13 08:04 historycache drwxr-xr-x 21 root root 4096 Nov 13 19:57 index drwxr-xr-x 6 root root 4096 Nov 13 19:57 xref root@46a6526c1312:/usr/local/tomcat# ls -la /var/opengrok/etc/ total 8 drwxr-xr-x 2 root root 4096 Nov 13 08:01 . drwxr-xr-x 1 root root 4096 Nov 13 08:01 ..

[tarzanek]

yeah, it should be timestamp, what files do you have in index? it seems the indexer never finished (which is also what your logs did say) ... so now to figure out why it didn't finish ... if you do ps -ef do you see any java running ?

[tarzanek]

also in etc there is no configuration.xml, which explains the error message you saw and confirms that indexer isn't done ...

[YoAm]

The container has been running for 2 days. Perhaps the cron job stops the indexing every time it is executed so it never gets to finish? I've used other OpenGrok docker images that use inotify, and they worked fine. I was looking forward to using the official image but it seems inferior.

[tarzanek]

mmmh, so that means that for your repo the image does something unexpected ... and yes, if you stop docker, the job never finishes ... how long does it take on the other docker ?

[tarzanek]

note that the other docker is for old opengrok - 0.12 official one is for 1.0 , so you might be hitting some bug ... btw. if you have lots of git renames that might be a prob (we saw an issue with 1.0 taking ages to figure out git renames for big repo)

[YoAm]

My repo is over 6GB in size. I've created working trees of separate branches from a single repo using git worktree. I never stopped docker, I was wondering if the indexing cron job stops the indexing before it completes. It would take a over an hour for the other container. We do have lots of git renames.

[tarzanek]

OPENGROK_RENAMED_FILES_HISTORY=off should be set for 1.0, we should patch this repo indexing cron job should finish, but I never tried it with a bigger repo (than OpenGroks), so there might be issues there, ideal case is to observe, debug and root cause why the indexing doesn't finish for you, but renames explain exactly what you see, so above variable set before the indexer will definitely get you going

[tarzanek]

I am talking about prefixing above here: https://github.com/OpenGrok/docker/blob/master/scripts/index.sh#L4

resp. above variable is from current master, so for 1.0 it'd be this: https://github.com/OpenGrok/OpenGrok/blob/9702bce5678e47213f5d33b40fa2a11f77914831/OpenGrok#L45

so patching the docker indexer script with prefix like "OPENGROK_DISABLE_RENAMED_FILES_HISTORY=true OpenGrok index ... " will get you going

[YoAm]

Is there a way to pass this through the docker run command?

[YoAm]

itszero\opengrok takes around an hour to index.

[tarzanek]

@YoAm well this is just a start for docker, so we will need to figure out if there is a way to pass such param through docker (would be cool)

itszero\opengrok is also on 0.12 , so no git renames there, 1.0 should be much faster (after you disable git renames, which block your case)

[YoAm]

I tried passing it using the docker run -e (environment variable) parameter but 22 hours later, it looks like it's still indexing. Seeing the following in the logs:

> $ sudo docker exec -it opengrok grep  -C 20 -r SEVERE /var/opengrok/
> /var/opengrok/log/opengrok0.7.log-2017-11-19 14:10:21.770+0000 FINE t1 Executor.registerErrorHandler: Installing default uncaught exception handler
> /var/opengrok/log/opengrok0.7.log:2017-11-19 14:10:23.638+0000 SEVERE t1 Indexer.main: Unexpected Exception
> /var/opengrok/log/opengrok0.7.log-java.net.ConnectException: Connection refused (Connection refused)
> /var/opengrok/log/opengrok0.7.log-      at java.net.PlainSocketImpl.socketConnect(Native Method)
> /var/opengrok/log/opengrok0.7.log-      at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> /var/opengrok/log/opengrok0.7.log-      at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> /var/opengrok/log/opengrok0.7.log-      at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> /var/opengrok/log/opengrok0.7.log-      at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> /var/opengrok/log/opengrok0.7.log-      at java.net.Socket.connect(Socket.java:589)
> /var/opengrok/log/opengrok0.7.log-      at java.net.Socket.connect(Socket.java:538)
> /var/opengrok/log/opengrok0.7.log-      at java.net.Socket.<init>(Socket.java:434)
> /var/opengrok/log/opengrok0.7.log-      at java.net.Socket.<init>(Socket.java:211)
> /var/opengrok/log/opengrok0.7.log-      at org.opensolaris.opengrok.configuration.messages.Message.write(Message.java:290)
> /var/opengrok/log/opengrok0.7.log-      at org.opensolaris.opengrok.index.Indexer.main(Indexer.java:595)
> /var/opengrok/log/opengrok0.7.log-
> --
> /var/opengrok/log/opengrok2.1.log-2017-11-20 11:20:02.258+0000 FINE t1 Executor.registerErrorHandler: Installing default uncaught exception handler
> /var/opengrok/log/opengrok2.1.log:2017-11-20 11:20:03.115+0000 SEVERE t1 Indexer.main: Unexpected Exception
> /var/opengrok/log/opengrok2.1.log-java.net.ConnectException: Connection refused (Connection refused)
> /var/opengrok/log/opengrok2.1.log-      at java.net.PlainSocketImpl.socketConnect(Native Method)
> /var/opengrok/log/opengrok2.1.log-      at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> /var/opengrok/log/opengrok2.1.log-      at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> /var/opengrok/log/opengrok2.1.log-      at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> /var/opengrok/log/opengrok2.1.log-      at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> /var/opengrok/log/opengrok2.1.log-      at java.net.Socket.connect(Socket.java:589)
> /var/opengrok/log/opengrok2.1.log-      at java.net.Socket.connect(Socket.java:538)
> /var/opengrok/log/opengrok2.1.log-      at java.net.Socket.<init>(Socket.java:434)
> /var/opengrok/log/opengrok2.1.log-      at java.net.Socket.<init>(Socket.java:211)
> /var/opengrok/log/opengrok2.1.log-      at org.opensolaris.opengrok.configuration.messages.Message.write(Message.java:290)
> /var/opengrok/log/opengrok2.1.log-      at org.opensolaris.opengrok.index.Indexer.main(Indexer.java:595)
> /var/opengrok/log/opengrok2.1.log-
> --
> /var/opengrok/log/opengrok0.1.log-2017-11-20 11:30:02.270+0000 FINE t1 Executor.registerErrorHandler: Installing default uncaught exception handler
> /var/opengrok/log/opengrok0.1.log:2017-11-20 11:30:03.095+0000 SEVERE t1 Indexer.main: Unexpected Exception
> /var/opengrok/log/opengrok0.1.log-java.net.ConnectException: Connection refused (Connection refused)
> /var/opengrok/log/opengrok0.1.log-      at java.net.PlainSocketImpl.socketConnect(Native Method)
> /var/opengrok/log/opengrok0.1.log-      at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> /var/opengrok/log/opengrok0.1.log-      at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> /var/opengrok/log/opengrok0.1.log-      at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> /var/opengrok/log/opengrok0.1.log-      at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> /var/opengrok/log/opengrok0.1.log-      at java.net.Socket.connect(Socket.java:589)
> /var/opengrok/log/opengrok0.1.log-      at java.net.Socket.connect(Socket.java:538)
> /var/opengrok/log/opengrok0.1.log-      at java.net.Socket.<init>(Socket.java:434)
> /var/opengrok/log/opengrok0.1.log-      at java.net.Socket.<init>(Socket.java:211)
> /var/opengrok/log/opengrok0.1.log-      at org.opensolaris.opengrok.configuration.messages.Message.write(Message.java:290)
> /var/opengrok/log/opengrok0.1.log-      at org.opensolaris.opengrok.index.Indexer.main(Indexer.java:595)
> /var/opengrok/log/opengrok0.1.log-
> --
> /var/opengrok/log/opengrok1.1.log-2017-11-20 11:25:02.699+0000 FINE t1 Executor.registerErrorHandler: Installing default uncaught exception handler
> /var/opengrok/log/opengrok1.1.log:2017-11-20 11:25:03.655+0000 SEVERE t1 Indexer.main: Unexpected Exception
> /var/opengrok/log/opengrok1.1.log-java.net.ConnectException: Connection refused (Connection refused)
> /var/opengrok/log/opengrok1.1.log-      at java.net.PlainSocketImpl.socketConnect(Native Method)
> /var/opengrok/log/opengrok1.1.log-      at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> /var/opengrok/log/opengrok1.1.log-      at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> /var/opengrok/log/opengrok1.1.log-      at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> /var/opengrok/log/opengrok1.1.log-      at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> /var/opengrok/log/opengrok1.1.log-      at java.net.Socket.connect(Socket.java:589)
> /var/opengrok/log/opengrok1.1.log-      at java.net.Socket.connect(Socket.java:538)
> /var/opengrok/log/opengrok1.1.log-      at java.net.Socket.<init>(Socket.java:434)
> /var/opengrok/log/opengrok1.1.log-      at java.net.Socket.<init>(Socket.java:211)
> /var/opengrok/log/opengrok1.1.log-      at org.opensolaris.opengrok.configuration.messages.Message.write(Message.java:290)
> /var/opengrok/log/opengrok1.1.log-      at org.opensolaris.opengrok.index.Indexer.main(Indexer.java:595)
> /var/opengrok/log/opengrok1.1.log-

[tarzanek]

can you fork this repo, change it directly and retry? I will try to find time and play with this, ev. change the repo myself (but I don't promise anything, it might very well be I will have time for this only during Christmas ... )

[fenglintianxia6]

Any solutions? I have the same proble.

There was an error!
CONFIGURATION parameter has not been configured in web.xml! Please configure your webapp.
Unable to determine source root path. Missing configuration?

"http://localhost:8080" gives an error above.

And I don't find '/var/opengrok/log'location ,No any log docker run -d -v /Volumes/android/aosp/:/src -p 8080:8080 opengrok/docker:latest this is the command I used.

[vladak]

You will not find any logs because the indexer in the docker image is set up (via the logging.properties file) not to generate logs to file. It should generate logs to the console, though.

[vladak]

@fenglintianxia6 is the indexer still running perhaps ? (given you seem to be indexing AOSP which normally can take even days to index if you have not provisioned your docker instance with enough resources)

[vladak]

I think the main problem is user perception rather than technical problem which stems from the fact that until the initial index is done the webapp reports missing configuration.

This can be approached (after oracle/opengrok#2600 is merged in) by running /scripts/index.sh with added --noIndex option right after the Tomcat is up and before the initial index. This way the webapp will no longer report missing configuration, it will merely display empty list of projects.

Still, with the docker images being often treated as appliances, this might not be enough so we might want to insert a text into the main page body saying the initial indexing is in progress and remove it after the initial index is done.