jonaswinkler / paperless-ng

A supercharged version of paperless: scan, index and archive all your physical documents
https://paperless-ng.readthedocs.io/en/latest/
GNU General Public License v3.0
5.38k stars 355 forks source link

Polling doesn't seem to work #1159

Open jacobtomlinson opened 3 years ago

jacobtomlinson commented 3 years ago

I am running paperless-ng 1.4.5 in a container. The consume directory is mounted via NFS so inotify does not work. I have set PAPERLESS_CONSUMER_POLLING=60 with the expectation that the directory will be polled every 60 seconds instead.

However new documents added to the directory do not seem to be found. If I restart the container all documents in the directory are successfully imported so the first launch check is working and import is working.

Cantello commented 3 years ago

Same here with the same container, only with PAPERLESS_CONSUMER_POLLING=120 and the consume folder is on an SMB server. No error listed in the log and after a container restart, everything works.

grundsch commented 3 years ago

I'm running paperless-ng 1.4.5 in docker with docker-compose, inbox folder is an NFS share mounted on the host. I've added the PAPERLESS_CONSUMER_POLLING: 30 in the docker-compose.yml It's actually working fine, except when I'm scanning a large file that takes some time to be written on the NFS share. In that case, paperless does not index it (multiple errors in the logs - paperless waits for the file to stop changing size, but I think my scanner took more time to actually write the first/next/last page, and paperless got impatient). At that point it will ignore it and will not index it later. If I rename the file, then it is detected and processed correctly. Not sure if it's related to your scenario, but it may give you one more datapoint to look for errors...

dunklesToast commented 3 years ago

Could you post your docker-compose.yml?

Cantello commented 3 years ago

Could you post your docker-compose.yml?

Mine would be:


    image: redis:6.0
    restart: unless-stopped
    container_name: redis_paperless
    networks:
      - proxy

  paperless:
    image: jonaswinkler/paperless-ng:latest
    restart: always
    container_name: paperless
    networks:
      - proxy
    depends_on:
      - broker
    ports:
      - 8000:8000
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000"]
      interval: 30s
      timeout: 10s
      retries: 5
    volumes:
      - /opt/paperless/data:/usr/src/paperless/data
      - /mnt/Backup/Documents:/usr/src/paperless/media
      - /opt/paperless/export:/usr/src/paperless/export
      - /mnt/Stuff/_Scans:/usr/src/paperless/consume
    env_file: /opt/paperless/paperless.env
    environment:
      PAPERLESS_REDIS: redis://broker:6379```
grundsch commented 3 years ago

here's mine:

version: "3.4"
services:
  broker:
    image: redis:6.0
    restart: unless-stopped

  db:
    image: postgres:13
    restart: unless-stopped
    volumes:
      - pgdata:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: paperless
      POSTGRES_USER: paperless
      POSTGRES_PASSWORD: paperless

  webserver:
    image: jonaswinkler/paperless-ng:latest
    restart: unless-stopped
    depends_on:
      - db
      - broker
      - gotenberg
      - tika
    ports:
      - 8000:8000
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000"]
      interval: 30s
      timeout: 10s
      retries: 5
    volumes:
      - /mnt/public/paperless/data:/usr/src/paperless/data
      - /mnt/public/paperless/media:/usr/src/paperless/media
      - ./export:/usr/src/paperless/export
      - /mnt/public/paperless_inbox:/usr/src/paperless/consume
    env_file: docker-compose.env
    environment:
      PAPERLESS_REDIS: redis://broker:6379
      PAPERLESS_DBHOST: db
      PAPERLESS_TIKA_ENABLED: 1
      PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000
      PAPERLESS_TIKA_ENDPOINT: http://tika:9998
      PAPERLESS_CONSUMER_POLLING: 30

  gotenberg:
    image: thecodingmachine/gotenberg
    restart: unless-stopped
    environment:
      DISABLE_GOOGLE_CHROME: 1

  tika:
    image: apache/tika
    restart: unless-stopped

volumes:
  data:
  media:
  pgdata:
dunklesToast commented 3 years ago

Hmm. I had the same issue a week ago. What helped was to pin the image to jonaswinkler/paperless-ng:1.4.5 and to move everything from docker-compose.env to the docker-compose.yml's environment section. Maybe that'll help you as well

smseidl commented 3 years ago

Has anyone had luck resolving this? Did dunklesToast comment work for anyone else?

ihatemyisp commented 2 years ago

Has anyone had luck resolving this? Did dunklesToast comment work for anyone else?

I moved the environment variables from the .env to the .yml and things have been running smoother for me.