Intermittent errors appearing for events dating a few weeks back, current events do not upload

[smshapiro85]

• Unifi Protect Backup version: latest
• Unifi Protect version: 3.0.26
• Python version: 3.11.8
• Operating System: linux
• Are you using a docker container or native?: Docker
• Rclone backup location: Dropbox

Description

Scratching my head a bit on this- I am getting intermittent errors with events from 16-17 days ago (retention is set to 21d).

Here is a screenshot of a few examples of the errors. They appear to come in waves at various times throughout the day, and are errors for events that happened a while back.

Today's events do not appear to be successfully uploading either, so I am not sure if there is a backlog and I am getting rate limited with dropbox, or something else.

The only clue I have is that I was previously limiting detection types to "person, vehicle, ring" and did not appear to have any issues like this. Once I removed that restriction to allow backup of my G3 Instant cameras that do not have smart detections, I started getting these errors.

Here is my Docker compose:

version: "3.7"
services:
  unifi-protect-backup:
    image: ghcr.io/ep1cman/unifi-protect-backup
    container_name: unifi-protect-backup
    environment:
      UFP_USERNAME: unifi-protect-backup
      UFP_PASSWORD: XXXXXXXXX
      UFP_ADDRESS: 192.168.1.1
      UFP_SSL_VERIFY: false
      RCLONE_DESTINATION: XXXXXXX:unifi-protect-backup-2
      RCLONE_RETENTION: 21d
      APPRISE_NOTIFIERS: ERROR=pover://XXXXXXX@XXXXXX
    volumes:
      - /volume2/dockerssd/unifi-protect-backup:/data
      - /volume2/dockerssd/unifi-protect-backup:/config/database/
      - type: bind
        source: /volume2/dockerssd/unifi-protect-backup/config/rclone/rclone.conf
        target: /config/rclone/rclone.conf
    restart: always

[ep1cman]

It looks like the issue here is with rclone. I would recommend trying to upload some files with rclone outside of the container and checking the rclone forums for similar issues. Its odd that rclone hasn't returned any messages about why it has failed

[smshapiro85]

Hey thank you for your reply. I think I may have figured it out but I will reply back later to confirm.

I was looking at the logs and noticed the video download buffer was maxed out, the number of videos in the queue was hovering around 80, and the logs were still showing videos uploading from a number of weeks back, so I increased the video download buffer to 10GiB and restarted.

There are now over 40,000 events in the queue, and the video download buffer is ~ 4.5GiB/10GiB, so I think when I updated my config a few days ago to include all motion events, and not just smart detections, the app had to go back and upload tens of thousands of events and it was overrunning the buffer, potentially resulting in rclone errors.

[ep1cman]

The download buffer is handled externally to rclone so one should not effect the other, but you might be being rate limited by dropbox or something similar. The download buffer is kept in RAM so please make sure your system actually has 10gb of ram available to it otherwise it will start swapping to disk and things will grind to a halt

[smshapiro85]

Ah. I see. My NAS has indeed grinded down to a halt. Updating to 1GiB and restarting.

[smshapiro85]

Sigh, I think the root cause was that there was a permission issue with my folders that was preventing the app from creating the temporary rclone files. Closing the issue. Thank you again for your help.