Can't autodeploy application xar application with ft-client dep

[gmella]

Hi, Please find attached an application that is not properly-deployed using next line:

docker run -p 8080:8080 -v $PWD/build/test-ft-client-autodeploy-0.1.xar:/exist/autodeploy/test-ft-client-autodeploy-0.1.xar existdb/existdb:4.5.0

If you comment the ft-client dependency line in the expath-pkg.xml, the deployement works. Then you need to install on the existdb instance the ft-client (using package manager or xquery cmd)

Hey ! Github does not yet know xar extension ;) I then replace the file name extension from .xar to .zip test-ft-client-autodeploy-0.1.zip

Hope you can make that more flexible than what it already is ;)

Big thanks in advance,

Guillaume

[duncdrum]

Hi, a general note on your expath-pkg.xml you should add @semver-min to your dependency declarations.

Before I go into full debugging mode, can you see if the following works:

docker pull existdb/existdb:4.5.0
docker create  --name exist-ft -p 8080:8080 existdb/existdb:4.5.0
docker cp ./build/test-ft-client-autodeploy-0.1.xar exist-ft:exist/autodeploy
docker start exist-ft

[gmella]

I tried with both xar modes and the behaviour is the same.

[grantmacken]

@duncdrum @gmella

the problem outline as I understand it is... if a user copies a xar into the autodeploy dir. e.g.

docker cp ./test.xar exist:/autodeploy

then when the user runs exist container the xar will not auto deploy.

Docker images by their nature are “ephemeral” Without creating bound mount volumes, you will get a fresh exist instance every time you run the container.

The provided docker-compose file, mounts the config dir and the exist data dir We do this so any changes to both directories contents will persist across reboots.

The autodeploy is not bound so any changes to it, will not persist.

Solutions. @gmella

1. stop exist. docker-compose down
2. look at the example (docker-compose file)'[https://github.com/eXist-db/docker-existdb/blob/develop/docker-compose.yml]
3. add an new autodeploy volume to (docker-compose file)'[https://github.com/eXist-db/docker-existdb/blob/develop/docker-compose.yml]
4. start exist docker-compose up -d
5. docker cp your xar into the autodeploy dir
6. docker-compose down
7. docker-compose up -d

---

@duncdrum

docker pull existdb/existdb:4.5.0
docker create  --name exist-ft -p 8080:8080 existdb/existdb:4.5.0
docker cp ./build/test-ft-client-autodeploy-0.1.xar exist-ft:exist/autodeploy
docker start exist-ft

that doesn't look right, I think what you are looking for is docker commit

1. run the exist container you can just use docker-compose up -d
2. copy the xar over docker cp ./build/test-ft-client-autodeploy-0.1.xar exist:exist/autodeploy
3. commit as new image docker container commit exist myNewImageName
4. check! docker images You should now have a new docker image myNewImageName with the xar in the autodeploy dir.

If you want to bring run this new image using docker-compose

1. stop your existing image. docker-compose down
2. run docker images , note your new 'repository' image name
3. adjust your .env file

   DOCKER_IMAGE=myNewImageName
   DOCKER_TAG=latest

4. run docker-compose up -d

[gmella]

Hi @grantmacken, I already use some doker-compose to autodeploy application putting them in a volume linked to /exist/autodeploy and this works like a charm even if this is really simple (and ephemeral). As I told, the basic applications are properly installed. But trying to deploy before exist server has started and is ready for request does not work.

I guess that is is not related to docker because ft-client embeds some crypt jar that may be in conflict with exist main distribution. I not able to perform debug and appreciate your help. If that can help, here is the error thrown when ft-client module is installed in the post-install.xql step: org.exist.xquery.XQueryContext.getDocumentBuilder()Lorg/exist/memtree/MemTreeBuilder

Regards,

[grantmacken]

@gmella If you have successfully deployed other apps from autodeploy dir, (using a linked mount ) then the app that fails to deploy is most likely at fault and this is not a eXist docker image issue.

[duncdrum]

@grantmacken actually no, docker commit is for storing modification to a running container. The sequence of docker create, docker cp, docker start will have the desired effect of persistently storing the file inside autodeploy folder across reboots, just as using FROM, COPY in a dockerfile. It is the recommended way of doing this, since it is the only way to ensure that exist scans the folder upon start-up.

@gmella there are some well known but undocumented quirks to the autodeploy folder that have nothing to do with docker. Until we can fix these within exist, my recommendation is to copy both the ft-client.xar and your app.xar to autodeploy, while making sure that the name of your app's xar is alphabetically sorted later than the ft-flient.xar. The content of autodeploy are processed in alphabetical order, irrespective of dependencies declared for each xar.

Problems with crypto jar, would show when you install your app via package-manager, since installing the app works via GUI, the problems are with exist's autodeploy code. If however, having the following in your expath-pkg.xml

<dependency package="http://expath.org/ns/ft-client"/>

doesn't work when you install your app via package-manager. Then the problem is with the ft-client module.

[grantmacken]

@duncdrum interesting never done that way, however the commit => new image approach works

commit ...

Create a new image from a container’s changes ... It can be useful to commit a container’s file changes or settings into a new image

[gmella]

OK, thank you for your help, I already used the workarround to put deps's xar in the autodeploy. I guess that alphabetical order does not matter if the application module does not use the code durint post/pre install steps. Anyway I learned some nice steps using docker and will try to test another deployement with another pure xquery dependency to compare. Then I may reopen this issue with ft-client in the main exist repo side. Cheers,