stain / jena-docker

Docker image for Apache Jena riot
Apache License 2.0
99 stars 86 forks source link

issue when using a volume for data #35

Closed eriknl1982 closed 3 years ago

eriknl1982 commented 4 years ago

Hi,

I have an issue with the jena-fuseki docker image.

I have the following configuration in my docker-compose file:

jena-fuseki: image: stain/jena-fuseki:latest expose:

  • 3030 environment: ADMIN_PASSWORD: ${JENA_FUSEKI_ADMIN_PASSWORD} volumes:
  • jena-fuseki_data:/fuseki

When I start this container for the first time, it works without any issues. When I stop the container and start it for a second time, I get the following errors in the log:

jena-fuseki_1 | [2020-04-02 11:11:25] TDB WARN Your platform does not support checking process liveness so TDB disk locations cannot be reliably locked to prevent possible corruption due to unsafe multi-JVM usage jena-fuseki_1 | java.io.IOException: Cannot run program "ps": error=2, No such file or directory jena-fuseki_1 | at java.base/java.lang.ProcessBuilder.start(Unknown Source) jena-fuseki_1 | at java.base/java.lang.ProcessBuilder.start(Unknown Source) jena-fuseki_1 | at org.apache.jena.tdb.sys.ProcessUtils.getProcessInfo(ProcessUtils.java:144) jena-fuseki_1 | at org.apache.jena.tdb.sys.ProcessUtils.isAlive(ProcessUtils.java:85) jena-fuseki_1 | at org.apache.jena.tdb.base.file.LocationLock.canObtain(LocationLock.java:148) jena-fuseki_1 | at org.apache.jena.tdb.StoreConnection._makeAndCache(StoreConnection.java:280) jena-fuseki_1 | at org.apache.jena.tdb.StoreConnection.make(StoreConnection.java:244) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.init$(SystemState.java:89) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.init(SystemState.java:56) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.getDataset(SystemState.java:45) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiWebapp.initializeDataAccessPoints(FusekiWebapp.java:217) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiServerListener.serverInitialization(FusekiServerListener.java:93) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiServerListener.contextInitialized(FusekiServerListener.java:53) jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:952) jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:558) jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:917) jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:370) jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497) jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459) jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:847) jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:287) jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545) jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138) jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108) jena-fuseki_1 | at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) jena-fuseki_1 | at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:410) jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138) jena-fuseki_1 | at org.eclipse.jetty.server.Server.start(Server.java:416) jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108) jena-fuseki_1 | at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) jena-fuseki_1 | at org.eclipse.jetty.server.Server.doStart(Server.java:383) jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) jena-fuseki_1 | at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.start(JettyFusekiWebapp.java:138) jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:372) jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356) jena-fuseki_1 | at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) jena-fuseki_1 | at jena.cmd.CmdMain.mainRun(CmdMain.java:58) jena-fuseki_1 | at jena.cmd.CmdMain.mainRun(CmdMain.java:45) jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104) jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67) jena-fuseki_1 | Caused by: java.io.IOException: error=2, No such file or directory jena-fuseki_1 | at java.base/java.lang.ProcessImpl.forkAndExec(Native Method) jena-fuseki_1 | at java.base/java.lang.ProcessImpl.(Unknown Source) jena-fuseki_1 | at java.base/java.lang.ProcessImpl.start(Unknown Source) jena-fuseki_1 | ... 42 more jena-fuseki_1 | [2020-04-02 11:11:25] Server ERROR Exception in initialization: Process ID 8 can't open database at location /fuseki/system/ because it is already locked by the process with PID 7. TDB databases do not permit concurrent usage across JVMs so in order to prevent possible data corruption you cannot open this location from the JVM that does not own the lock for the dataset jena-fuseki_1 | [2020-04-02 11:11:25] WebAppContext WARN Failed startup of context o.e.j.w.WebAppContext@4e61e4c2{Apache Jena Fuseki Server,/,file:///jena-fuseki/webapp/,UNAVAILABLE} jena-fuseki_1 | org.apache.jena.tdb.TDBException: Process ID 8 can't open database at location /fuseki/system/ because it is already locked by the process with PID 7. TDB databases do not permit concurrent usage across JVMs so in order to prevent possible data corruption you cannot open this location from the JVM that does not own the lock for the dataset jena-fuseki_1 | at org.apache.jena.tdb.StoreConnection._makeAndCache(StoreConnection.java:282) jena-fuseki_1 | at org.apache.jena.tdb.StoreConnection.make(StoreConnection.java:244) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.init$(SystemState.java:89) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.init(SystemState.java:56) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.SystemState.getDataset(SystemState.java:45) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiWebapp.initializeDataAccessPoints(FusekiWebapp.java:217) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiServerListener.serverInitialization(FusekiServerListener.java:93) jena-fuseki_1 | at org.apache.jena.fuseki.webapp.FusekiServerListener.contextInitialized(FusekiServerListener.java:53) jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:952) jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:558) jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:917) jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:370) jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497) jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459) jena-fuseki_1 | at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:847) jena-fuseki_1 | at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:287) jena-fuseki_1 | at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545) jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138) jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108) jena-fuseki_1 | at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) jena-fuseki_1 | at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:410) jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138) jena-fuseki_1 | at org.eclipse.jetty.server.Server.start(Server.java:416) jena-fuseki_1 | at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108) jena-fuseki_1 | at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) jena-fuseki_1 | at org.eclipse.jetty.server.Server.doStart(Server.java:383) jena-fuseki_1 | at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) jena-fuseki_1 | at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.start(JettyFusekiWebapp.java:138) jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:372) jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356) jena-fuseki_1 | at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) jena-fuseki_1 | at jena.cmd.CmdMain.mainRun(CmdMain.java:58) jena-fuseki_1 | at jena.cmd.CmdMain.mainRun(CmdMain.java:45) jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104) jena-fuseki_1 | at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67) jena-fuseki_1 | [2020-04-02 11:11:25] Server INFO Started 2020/04/02 11:11:25 UTC on port 3030

the problem seems to be the file /fuseki/system/tdb.lock. Whenever this file is present, the above error is triggered. When I remove the file, the container starts normally again.

it seems to me that on shutdown of the container, this file should be cleaned up. Is there any fix or workaround for this issue?

dpnevmatikos commented 4 years ago

Hi, Same issue here.

  1. Created the container with :
    docker run  -p 3030:3030 -e ADMIN_PASSWORD=pw123 stain/jena-fuseki
  2. Uploaded a new dataset (ttl file)
  3. Stopped the container with: docker stop <name>
  4. Started the container with docker start <name>

and the issue appeared:

[2020-04-03 09:27:09] Server     ERROR Exception in initialization: caught: Process ID 7 can't open database at location /fuseki/databases/soda/ because it is already locked by the process with PID 8. TDB databases do not permit concurrent usage across JVMs so in order to prevent possible data corruption you cannot open this location from the JVM that does not own the lock for the dataset
[2020-04-03 09:27:09] WebAppContext WARN  Failed startup of context o.e.j.w.WebAppContext@329a1f8d{Apache Jena Fuseki Server,/,file:///jena-fuseki/webapp/,UNAVAILABLE}
org.apache.jena.assembler.exceptions.AssemblerException: caught: Process ID 7 can't open database at location /fuseki/databases/soda/ because it is already locked by the process with PID 8. TDB databases do not permit concurrent usage across JVMs so in order to prevent possible data corruption you cannot open this location from the JVM that does not own the lock for the dataset
  doing:
    root: http://base/#tdb_dataset_readwrite with type: http://jena.hpl.hp.com/2008/tdb#DatasetTDB assembler class: class org.apache.jena.tdb.assembler.DatasetAssemblerTDB

        at org.apache.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.openBySpecificType(AssemblerGroup.java:165)
        at org.apache.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.open(AssemblerGroup.java:144)
        at org.apache.jena.assembler.assemblers.AssemblerGroup$ExpandingAssemblerGroup.open(AssemblerGroup.java:93)
        at org.apache.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:39)
        at org.apache.jena.assembler.assemblers.AssemblerBase.open(AssemblerBase.java:35)
        at org.apache.jena.fuseki.build.FusekiConfig.getDataset(FusekiConfig.java:638)
        at org.apache.jena.fuseki.build.FusekiConfig.buildDataService(FusekiConfig.java:443)
        at org.apache.jena.fuseki.build.FusekiConfig.buildDataAccessPoint(FusekiConfig.java:433)
        at org.apache.jena.fuseki.build.FusekiConfig.readConfiguration(FusekiConfig.java:416)
        at org.apache.jena.fuseki.build.FusekiConfig.readConfigurationDirectory(FusekiConfig.java:395)
        at org.apache.jena.fuseki.webapp.FusekiWebapp.initializeDataAccessPoints(FusekiWebapp.java:216)
        at org.apache.jena.fuseki.webapp.FusekiServerListener.serverInitialization(FusekiServerListener.java:93)
        at org.apache.jena.fuseki.webapp.FusekiServerListener.contextInitialized(FusekiServerListener.java:53)
        at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:952)
        at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:558)
        at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:917)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:370)
        at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1497)
        at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1459)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:847)
        at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:287)
        at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:545)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
        at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:410)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:138)
        at org.eclipse.jetty.server.Server.start(Server.java:416)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:108)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
        at org.eclipse.jetty.server.Server.doStart(Server.java:383)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
        at org.apache.jena.fuseki.cmd.JettyFusekiWebapp.start(JettyFusekiWebapp.java:138)
        at org.apache.jena.fuseki.cmd.FusekiCmd.runFuseki(FusekiCmd.java:372)
        at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.exec(FusekiCmd.java:356)
        at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
        at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
        at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
        at org.apache.jena.fuseki.cmd.FusekiCmd$FusekiCmdInner.innerMain(FusekiCmd.java:104)
        at org.apache.jena.fuseki.cmd.FusekiCmd.main(FusekiCmd.java:67)
Caused by: org.apache.jena.tdb.TDBException: Process ID 7 can't open database at location /fuseki/databases/soda/ because it is already locked by the process with PID 8. TDB databases do not permit concurrent usage across JVMs so in order to prevent possible data corruption you cannot open this location from the JVM that does not own the lock for the dataset
        at org.apache.jena.tdb.StoreConnection._makeAndCache(StoreConnection.java:282)
        at org.apache.jena.tdb.StoreConnection.make(StoreConnection.java:244)
        at org.apache.jena.tdb.StoreConnection.make(StoreConnection.java:258)
        at org.apache.jena.tdb.transaction.DatasetGraphTransaction.<init>(DatasetGraphTransaction.java:69)
        at org.apache.jena.tdb.sys.TDBMaker.createDirect(TDBMaker.java:126)
        at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(Unknown Source)
        at org.apache.jena.tdb.sys.TDBMaker._create(TDBMaker.java:112)
        at org.apache.jena.tdb.sys.TDBMaker.createDatasetGraphTransaction(TDBMaker.java:43)
        at org.apache.jena.tdb.TDBFactory._createDatasetGraph(TDBFactory.java:93)
        at org.apache.jena.tdb.TDBFactory.createDatasetGraph(TDBFactory.java:71)
        at org.apache.jena.tdb.assembler.DatasetAssemblerTDB.make(DatasetAssemblerTDB.java:57)
        at org.apache.jena.tdb.assembler.DatasetAssemblerTDB.createDataset(DatasetAssemblerTDB.java:48)
        at org.apache.jena.sparql.core.assembler.DatasetAssembler.open(DatasetAssembler.java:43)
        at org.apache.jena.assembler.assemblers.AssemblerGroup$PlainAssemblerGroup.openBySpecificType(AssemblerGroup.java:157)
        ... 41 more
[2020-04-03 09:27:09] Server     INFO  Started 2020/04/03 09:27:09 UTC on port 3030

It appears that Jena server is not shutting down gracefully. Am I doing something wrong?

Update: Just noticed this solution and it seems to work.

stain commented 4 years ago

Thanks, I am able to reproduce. In a sense there are two issues:

The entrypoint currently also does the optional loading after server is started, which is why the exec goes into background..

inquota commented 4 years ago

Thanks for your explanation!

Is there a way for hook into this proces with the docker-entrypoint.sh and remove thetdb.lock before the container goes down?

namedgraph commented 4 years ago
  • Is this because docker-entrypoint.sh puts the JVM in the background with & and so the signal is not passed along?

You can save the PID from $! after the background command. But then you'll have to manually monitor its state, and ideally trap kill signals in the entrypoint script and pass them to the background process.

We tried a similar approach but abandoned it due to the complexity: https://github.com/AtomGraph/fuseki-docker/blob/fuseki-tdb/entrypoint.sh

thibaud-sanchez commented 4 years ago

This is the same issue as https://github.com/stain/jena-docker/issues/41 This is a problem with missing ps java.io.IOException: Cannot run program "ps": error=2, No such file or directory

I fixed it by creating my own DockerFIle which install procps and now when I stop my container lock files are removed :

FROM stain/jena-fuseki

RUN apt-get update; \
    apt-get install -y --no-install-recommends procps

There is a PR waiting merge : https://github.com/stain/jena-docker/pull/40

sroze commented 3 years ago

@stain FYI this issue can be closed.