CERT-Polska / karton

Distributed malware processing framework based on Python, Redis and S3.
https://karton-core.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
381 stars 45 forks source link

Issue with redis #254

Open bormaa opened 4 months ago

bormaa commented 4 months ago

Suddenly The data in redis becomes empty and karton dashboard looks like new however their were tasks their. I use this docker configuration for redis

image: redis/redis-stack-server:latest
ports:
  - "6379:6379"
restart: always
healthcheck:
  test: ["CMD", "redis-cli", "ping"]
  interval: 10s
  timeout: 3s
  retries: 10

I checked part of the logs and found this if it can help

209144:C 14 Apr 2024 20:47:31.079 Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB 8:M 14 Apr 2024 20:47:31.168 Background saving terminated with success 8:M 14 Apr 2024 20:52:14.504 DB saved on disk 8:M 14 Apr 2024 20:52:15.174 Got a flush started event 8:M 14 Apr 2024 20:52:15.178 DB saved on disk 8:M 14 Apr 2024 20:52:17.505 DB saved on disk 8:M 14 Apr 2024 20:52:18.172 DB saved on disk 8:M 14 Apr 2024 20:52:18.512 Got a flush started event 8:M 14 Apr 2024 20:52:18.524 DB saved on disk 8:M 14 Apr 2024 20:52:20.861 DB saved on disk 8:M 14 Apr 2024 20:57:22.039 100 changes in 300 seconds. Saving... 8:M 14 Apr 2024 20:57:22.045 Background saving started by pid 209492 209492:C 14 Apr 2024 20:57:22.053 DB saved on disk 209492:C 14 Apr 2024 20:57:22.055 Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB

psrok1 commented 4 months ago

If that's complete Docker configuration, you probably lacked volume on /data, so Redis database was stored directly in the container which may have been lost during restart.

See also:

bormaa commented 4 months ago

I edited my docker config to be

image: redis/redis-stack-server:latest
ports:
  - "6379:6379"
restart: always
healthcheck:
  test: ["CMD", "redis-cli", "ping"]
  interval: 10s
  timeout: 3s
  retries: 10
volumes:
  - ./redis-data:/data

The issue is still their suddenly finds the dashboard is empty however I am sure their is too many tasks that should take too much time