Simplified Deployment - ONE Docker Image for both Backend + Frontend

[ciur]

Problem

Current deployment, which is separated as backend docker image and frontend docker image is a pain to setup. In order to be marginally useful you need to create a docker compose with at least 3 services:

• backend (rest api backend)
• frontend (static assets i.e. js, css, html)
• traefik (routing between the backend and frontend)

I received lots of complains reported as bugs because of (usually traefik) misconfiguration. It makes perfect sense, since traefik is not the most intuitive tool to configure, but also it is difficult to reason in terms of routing between frontend and backend.

Also, there is another problem with above approach - the backend and frontend version should be compatible e.g does backend version 2.1.7 work with frontend version 2.1.2? Solution provided below will solve this problem as well.

Solution

Solution is relatively simple - is to copy static assets directly into backend, and let backend serve those static assets. In other words - instead of three docker images configuration (backend, frontend, traefik) you will be able to use only one - backend!

I am currently working on this.

[ciur]

Work in progress: https://github.com/papermerge/papermerge-core/pull/124

[ciur]

Please check here the new, simplified docker compose file. Use it only for papermerge/papermerge:2.1.8 docker image (and later).

Notice there is nothing about traefik, frontend/backend, web sockets in it - now there are just 4 services which, I guess, should make perfect sense to have around:

• rest api backend
• worker (for performing OCR)
• db
• redis

Now, the simplest way of lunching papermerge:

  docker run -p 16000:8000 \
      -e PAPERMERGE__MAIN__SECRET_KEY=abc \
      -e DJANGO_SUPERUSER_PASSWORD=123 \
      papermerge/papermerge:2.1.8

will include UI as well, on other words, if you point your browser to http://localhost:16000, you will see papermege login window.

If you experience any "gray screen", please either delete cookies for that domain (localhost:16000) or start in incognito mode.

Documentation update: work in progress....

[homehtha]

This is great, now I can login (it was impossible before, even in dev version it didn't work properly). Is OCR working in this version? I have uploaded one item in Inbox, but it keeps showing 0 and it wont ocr (gray circle). I also don't see an option to start ocr. Also drag'n'drop didn't work (should it?), I have uploaded file by manually choosing it via upload icon.

[ciur]

@homehtha did you start it with docker compose ? Is worker up and running ? Please provider here logs from both worker and backend as per docker compose

[kolossboss]

@ciur I also missing features. Cant Import and automates are missing too. I'm using docker compose to set this up.

version: '3.7'
x-backend: &backend
  image: papermerge/papermerge:2.1.8
  volumes:
    - /Volumes/Daten/Papermerge:/app/media
    - xapian_index:/app/xapian_index
  environment:
    - PAPERMERGE__MAIN__SECRET_KEY=${SECRET_KEY}
    - PAPERMERGE__DATABASE__TYPE=postgres
    - PAPERMERGE__DATABASE__USER=${DB_USER}
    - PAPERMERGE__DATABASE__NAME=${DB_NAME}
    - PAPERMERGE__DATABASE__PASSWORD=${DB_PASSWORD}
    - PAPERMERGE__DATABASE__HOST=${DB_HOST}
    - PAPERMERGE__REDIS__HOST=redis
    - PAPERMERGE__REDIS__PORT=6379
    - PAPERMERGE__SEARCH__PATH=/app/xapian_index
    - DJANGO_SUPERUSER_USERNAME=${SUPERUSER_USERNAME}
    - DJANGO_SUPERUSER_EMAIL=${SUPERUSER_EMAIL}
    - DJANGO_SUPERUSER_PASSWORD=${SUPERUSER_PASSWORD}
    - DJANGO_SETTINGS_MODULE=config.settings
services:
  worker:
    <<: *backend
    command: worker
  backend:
    <<: *backend
    ports:
      - '16000:8000'
  db:
    image: postgres:14.4
    volumes:
      - postgres_data:/var/lib/postgresql/data/
    environment:
      - POSTGRES_USER=${DB_USER}
      - POSTGRES_DB=${DB_NAME}
      - POSTGRES_PASSWORD=${DB_PASSWORD}
  redis:
    image: 'redis:6'
    ports:
      - '6379:6379'
    volumes:
      - redis_data:/data
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
volumes:
  postgres_data:
  media_root:
  xapian_index:
  redis_data:

Environment variables

DB_USER=postgres
DB_NAME=postgres
DB_PASSWORD=postgres
DB_HOST=db
DB_PORT=5432
SECRET_KEY=alsdkalsdjlaksdj90823423!KLKJLkjkjlkjlKLPOgrwqna
SUPERUSER_USERNAME=admin
SUPERUSER_EMAIL=admin@mail.com
SUPERUSER_PASSWORD=admin

The all 4 containers are running. Thats the log from worker

worker: Warm shutdown (MainProcess)
/venv/lib/python3.10/site-packages/requests/__init__.py:109: RequestsDependencyWarning: urllib3 (1.26.13) or chardet (None)/charset_normalizer (3.0.1) doesn't match a supported version!
  warnings.warn(
/venv/lib/python3.10/site-packages/celery/platforms.py:840: SecurityWarning: You're running the worker with superuser privileges: this is
absolutely not recommended!
Please specify a different user using the --uid option.
User information: uid=0 euid=0 gid=0 egid=0
  warnings.warn(SecurityWarning(ROOT_DISCOURAGED.format(

 -------------- worker-node-6ee313f29f18@papermerge v5.2.7 (dawn-chorus)
--- ***** ----- 
-- ******* ---- Linux-5.15.49-linuxkit-aarch64-with-glibc2.31 2023-03-26 15:15:01
- *** --- * --- 
- ** ---------- [config]
- ** ---------- .> app:         papermerge:0xffffb7e8dd50
- ** ---------- .> transport:   redis://redis:6379/0
- ** ---------- .> results:     disabled://
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> celery           exchange=papermerge(direct) key=papermerge

INFO 2023-03-26 15:15:01,702 connection Connected to redis://redis:6379/0
[2023-03-26 15:15:01,702: INFO/MainProcess] Connected to redis://redis:6379/0
INFO 2023-03-26 15:15:01,705 mingle mingle: searching for neighbors
[2023-03-26 15:15:01,705: INFO/MainProcess] mingle: searching for neighbors
INFO 2023-03-26 15:15:02,716 mingle mingle: all alone
[2023-03-26 15:15:02,716: INFO/MainProcess] mingle: all alone
INFO 2023-03-26 15:15:02,741 worker worker-node-6ee313f29f18@papermerge ready.
[2023-03-26 15:15:02,741: INFO/MainProcess] worker-node-6ee313f29f18@papermerge ready.

[ciur]

"import" and automates features were part of 2.0.x This new docker image ships 2.1.x version which does not include automates (will be re-added in later releases). Regarding imports. For 2.1.x version imports are performed with separate command line utility papermerge-cli