How to add plugins via Docker?

[gkoerk]

When running via Docker, the /plugin directory is not able to be bind mounted properly, as shaarli will not create the necessary files on startup. After normal startup, I was able to do:

docker cp -aL <container>:/var/www/shaarli/application/plugin /share/appdata/shaarli/plugin

and then add the following line to the docker-compose:

- /share/appdata/shaarli/plugin:/var/www/shaarli/application/plugin

The application starts up successfully after that. Will this workaround be sustainable?

[immanuelfodor]

My hack is to put everything inside the data folder in subfolders, then do some docker exec -it shaarli cp -rf from to to get them in place. Chmod/chown afterwards, of course.

[ArthurHoaro]

Hi, I use this method. Sorry for the French, but looking at the code, it might be understandable: extend the official Docker image, and clone the plugin I want to use.

If you guys have a better idea of how to easily include 3rd party plugins with the official Docker image, feel free to contribute.

[immanuelfodor]

This is pretty neat and docker-ish, much better than my scripted version, I might as well implement my own Dockerfile based on your example. Just waiting for confirmation if the material theme is working with the 0.11 versions at https://github.com/kalvn/Shaarli-Material/issues/94 then I'll definitely go for it, thanks!

[immanuelfodor]

Another idea that solves OP's problem: there could be a plugin-local folder, that could be used as bind-mount and could merge plugins with the base plugin folder, hence no custom docker images would be needed.

[ArthurHoaro]

This is a good idea, and it wouldn't be hard to add.

[immanuelfodor]

If it helps, I came up with this multistage Dockerfile in the meantime based on Arthur's above linked https://github.com/shaarli/Shaarli/issues/1372#issuecomment-544631701 solution but with improvements here and there:

FROM alpine/git as builder

RUN apk add --update --no-cache npm 

WORKDIR /tpl
RUN set -x \
    && git clone https://github.com/kalvn/Shaarli-Material.git material \
    && cd material \
    && npm install --unsafe-perm node-sass \
    && npm install \
    && npm run-script build

WORKDIR /plugins
RUN set -x \
    && git clone https://github.com/ArthurHoaro/code-coloration.git code_coloration \
    && git clone https://github.com/ilesinge/shaarli-related.git related \
    && git clone https://github.com/immanuelfodor/shaarli-descriptor.git shaarli_descriptor  \
    && git clone https://github.com/immanuelfodor/shaarli-markdown-toolbar.git markdown_toolbar \
    && git clone https://github.com/immanuelfodor/shaarli-custom-css.git custom_css \
    && git clone https://github.com/immanuelfodor/emojione.git emojione \
    && git clone https://github.com/kalvn/shaarli-plugin-autosave.git autosave \
    && git clone https://github.com/trailjeep/shaarli-favicons.git favicons \
    && rm -rf emojione/.git

FROM shaarli/shaarli:master

WORKDIR /var/www/shaarli
COPY --from=builder /tpl/material/material tpl/material
COPY --from=builder /plugins/code_coloration/code_coloration plugins/code_coloration
COPY --from=builder /plugins/related/related plugins/related
COPY --from=builder /plugins/shaarli_descriptor/shaarli_descriptor plugins/shaarli_descriptor
COPY --from=builder /plugins/markdown_toolbar/markdown_toolbar plugins/markdown_toolbar
COPY --from=builder /plugins/custom_css/custom_css plugins/custom_css
COPY --from=builder /plugins/emojione plugins/emojione
COPY --from=builder /plugins/autosave plugins/autosave
COPY --from=builder /plugins/favicons/favicons plugins/favicons

RUN set -x \
    && chown -R nginx. tpl plugins

EXPOSE 80

ENTRYPOINT ["/bin/s6-svscan", "/etc/services.d"]
CMD []

Place this docker-compose.yml next to it:

version: "3"

services:
  shaarli:
    build: .
    image: shaarli:master
    container_name: shaarli
    restart: unless-stopped
    ports:
     - "8080:80"
    volumes:
     - ./data:/var/www/shaarli/data
     - ./cache:/var/www/shaarli/cache

Starting it up is just docker-compose up -d --build

It might not be as ideal as a local plugins folder would be, but as I need to make a patch on an old plugin and build a template from source, it's even more convenient now.

Update 2020.03.18: Added the set -x and npm install --unsafe-perm node-sass commands to the Dockerfile to fix https://github.com/shaarli/Shaarli/issues/1372#issuecomment-600211746 via https://github.com/sass/node-sass/issues/2773#issuecomment-559039446

Update 2020.08.30: Added my new Custom CSS plugin to the list and also replaced emojione with my fork as the old needed some bash magic to make it work.

Update 2021.01.17: Removed gulp, and changed the template build command due to https://github.com/kalvn/Shaarli-Material/issues/128

[virtualtam]

+1 for adding a user-specific plugin directory, that could either be used as a dedicated volume for manual/scripted installation, or by user-built Docker images extending the official images.

[frenchvandal]

Hi, I run (successfully) a Shaarli instance with Docker. I wanted to install the Material theme, using @immanuelfodor method. So I cleaned the Dockerfile a bit, to that:

FROM alpine/git as builder

RUN apk add --update --no-cache npm \
    && npm install -g gulp

WORKDIR /tpl
RUN git clone https://github.com/kalvn/Shaarli-Material.git material \
    && cd material \
    && npm install \
    && gulp build

FROM shaarli/shaarli:master

WORKDIR /var/www/shaarli

COPY --from=builder /tpl/material/material tpl/material

RUN chown -R nginx. tpl

EXPOSE 80

ENTRYPOINT ["/bin/s6-svscan", "/etc/services.d"]
CMD []

Then my docker-compose up -d --build fails. The logs are verbose, here are some excerpts:

Cloning into 'material'...

> node-sass@4.10.0 install /tpl/material/node_modules/node-sass
> node scripts/install.js

Downloading binary from https://github.com/sass/node-sass/releases/download/v4.10.0/linux_musl-x64-72_binding.node
Cannot download "https://github.com/sass/node-sass/releases/download/v4.10.0/linux_musl-x64-72_binding.node":

HTTP error 404 Not Found

Hint: If github.com is not accessible in your location
      try setting a proxy via HTTP_PROXY, e.g.

      export HTTP_PROXY=http://example.com:1234

or configure npm proxy via

      npm config set proxy http://example.com:8080

> node-sass@4.10.0 postinstall /tpl/material/node_modules/node-sass
> node scripts/build.js

Building: /usr/bin/node /tpl/material/node_modules/node-gyp/bin/node-gyp.js rebuild --verbose --libsass_ext= --libsass_cflags= --libsass_ldflags= --libsass_library=

gyp ERR! stack Error: Can't find Python executable "python", you can set the PYTHON env variable.
gyp ERR! stack     at PythonFinder.failNoPython (/tpl/material/node_modules/node-gyp/lib/configure.js:484:19)
gyp ERR! stack     at PythonFinder.<anonymous> (/tpl/material/node_modules/node-gyp/lib/configure.js:406:16)
gyp ERR! stack     at F (/tpl/material/node_modules/which/which.js:68:16)
gyp ERR! stack     at E (/tpl/material/node_modules/which/which.js:80:29)
gyp ERR! stack     at /tpl/material/node_modules/which/which.js:89:16
gyp ERR! stack     at /tpl/material/node_modules/isexe/index.js:42:5
gyp ERR! stack     at /tpl/material/node_modules/isexe/mode.js:8:5
gyp ERR! stack     at FSReqCallback.oncomplete (fs.js:158:21)
gyp ERR! System Linux 4.15.0-88-generic
gyp ERR! command "/usr/bin/node" "/tpl/material/node_modules/node-gyp/bin/node-gyp.js" "rebuild" "--verbose" "--libsass_ext=" "--libsass_cflags=" "--libsass_ldflags=" "--libsass_library="
gyp ERR! cwd /tpl/material/node_modules/node-sass
gyp ERR! node -v v12.15.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
Build failed with error code: 1
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.7 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.7: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! node-sass@4.10.0 postinstall: `node scripts/build.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the node-sass@4.10.0 postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-03-17T17_44_59_294Z-debug.log
ERROR: Service 'shaarli' failed to build: The command '/bin/sh -c git clone https://github.com/kalvn/Shaarli-Material.git material     && cd material     && npm install     && npm update     && gulp build' returned a non-zero code: 1

Any idea on how I can solve this?

Thank you!

[immanuelfodor]

Maybe there is some dependency added to the Material theme that needs python, I'll build my image later to see what's happening.

[immanuelfodor]

It seems there's some alpine weirdness going on here, but managed to fix it via https://github.com/sass/node-sass/issues/2773#issuecomment-559039446

Updated the above Dockerfile with the set -x and npm install --unsafe-perm node-sass commands, now it builds like a charm, again.

[frenchvandal]

Thank you a lot @immanuelfodor 👍

I did not notice I removed the set -x when I trimmed the Dockerfile, my bad.

Indeed it works like a charm now, this theme is a game changer. I hope the maintainers will also include a /tpl directory when they update the Shaarli official image.

[frenchvandal]

@immanuelfodor I am very sorry to bother you again, I have another issue using the Material theme with this method: I cannot login once I have logout.

I get this message: You have been banned after too many failed login attempts. Try again later.

It seems to be an issue only using Material activated, when I switch back to the default theme (using a session on a different computer), the login page is OK.

Do you have any idea to solve this? No worries if you do not, I will just wait until the official image is plugin/template ready.

EDIT: OK no worries https://github.com/kalvn/Shaarli-Material/issues/95 https://github.com/shaarli/Shaarli/issues/1430

[immanuelfodor]

Yepp, it's the same for me, after building the docker, I've run into it: https://github.com/kalvn/Shaarli-Material/issues/95

[immanuelfodor]

My previous Shaarli docker build is from the master branch (dev version) 2 Oct @ f4c66259 and it's rock solid since then. You can easily switch to a previous working state by changing the second FROM clause to, e.g.:

FROM shaarli/shaarli:v0.11.1

I did not test it within docker, but it's supposed to work: https://github.com/kalvn/Shaarli-Material/issues/94

[lonix1]

@immanuelfodor Do you mind posting your complete working Dockerfile? Thanks!

[immanuelfodor]

Hi @lonix1, do you have any problem with the one in https://github.com/shaarli/Shaarli/issues/1372#issuecomment-546341830? I have one more plugin tailored to my hosting environment, but the file is basically the same as the one posted in the above comment. I've rebuilt it a few minutes ago just for sake of making sure everything is fine with latest master, and it looks good. Kalvn has also modified the Material theme since then to fix the login issue, if you build that file, it should be good.

[lonix1]

@immanuelfodor Oops sorry I saw bugs in later comments, I posted to wrong person. Just used it and it works!

[bushchannelstepper]

An easier workaround for me than building a new docker image. Excerpt from docker-compose:

volumes:
  - /mnt/docker/apps/shaarli-conf/data:/var/www/shaarli/data
  - /mnt/docker/apps/shaarli-conf/cache:/var/www/shaarli/cache
  - /mnt/docker/apps/shaarli-conf/tpl_material:/var/www/shaarli/tpl/material

[nodiscc]

So we have workarounds for this problem (either let Shaarli do its first startup and create the directory, then create a bind mount to the plugins directory, OR maintain a docker image which includes desired plugins). None of these are very practical. IMHO the long-term solution should be:

a plugin-local folder, that could be used as bind-mount and could merge plugins with the base plugin folder, hence no custom docker images would be needed.

[ArthurHoaro]

I agree, this comment has also been upvoted by a few people. Should we keep this for v0.12.0 or move it up one minor digit? It's not a major change, but still kind of a feature.