ep1cman / unifi-protect-backup

Python tool to backup unifi event clips in realtime
MIT License
577 stars 25 forks source link

Add ARM support for Docker image #45

Closed Smith8154 closed 1 year ago

Smith8154 commented 1 year ago

It would be nice to have an ARM version of the Docker container to run on things like Raspberry Pi's.

ep1cman commented 1 year ago

Given where I work this very much aught to be something I do 😂 I’ll look into it and let you know once I have the CI setup to build arm containers

ep1cman commented 1 year ago

Can you check the dev tag, it should now have an arm64 build

Smith8154 commented 1 year ago

Wow, that was fast! Guess it was pretty easy for you to put together 😄. I was able to build the container just fine, but I am getting an error when trying to start it. I translated the default Docker commands you provided to docker-compose.

Compose script:

  unifi-backup:
    container_name: unifi-backup
    image: ghcr.io/ep1cman/unifi-protect-backup:dev
    environment:
      - UFP_USERNAME='my-username'
      - UFP_PASSWORD='my-password
      - UFP_ADDRESS='my-ip'
      - UFP_SSL_VERIFY='false'
      - RCLONE_DESTINATION='backblaze-b2:/unifi_protect_backup'
      - RETENTION='14'
    volumes:
      - '/mnt/docker/unifi-protect-backup/data:/data'
      - '/mnt/docker/unifi-protect-backup/rclone.conf:/config/rclone.conf'

Error:

Usage: unifi-protect-backup [OPTIONS]
Try 'unifi-protect-backup --help' for help.
Smith8154 commented 1 year ago

Removing the single quotes around "false" fix that issue, but considering the rest of the options have single quotes around them, it's odd it is giving an error on just this one.

Running into another issue when trying to start it, but I believe this one is just because I'm stupid. It is having trouble finding the remote value, and I'm not sure how it should be formatted.

2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:  Config:
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    address="'10.50.0.1'"
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    port=443
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    username="'protect-backup'"
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    password=REDACTED
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    verify_ssl=False
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    rclone_destination='b2:/unifi_protect_backup'
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    retention='7d'
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    rclone_args=''
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    ignore_cameras=()
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    verbose=1
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    detection_types=['motion', 'person', 'vehicle', 'ring']
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:    file_structure_format='{camera_name}/{event.start:%Y-%m-%d}/{event.end:%Y-%m-%dT%H-%M-%S} {detection_type}.mp4'
2022-08-21 01:53:48 [INFO]:unifi_protect_backup.unifi_protect_backup:   Starting...
2022-08-21 01:53:48 [INFO]:unifi_protect_backup.unifi_protect_backup:   Checking rclone configuration...
2022-08-21 01:53:48 [DEBUG]:unifi_protect_backup.unifi_protect_backup:  rclone found: /usr/bin/rclone
Traceback (most recent call last):
  File "/usr/bin/unifi-protect-backup", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.10/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/usr/lib/python3.10/site-packages/unifi_protect_backup/cli.py", line 109, in main
    loop.run_until_complete(event_listener.start())
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/usr/lib/python3.10/site-packages/unifi_protect_backup/unifi_protect_backup.py", line 278, in start
    await self._check_rclone()
  File "/usr/lib/python3.10/site-packages/unifi_protect_backup/unifi_protect_backup.py", line 414, in _check_rclone
    raise ValueError(f"rclone does not have a remote called `{remote}`")
ValueError: rclone does not have a remote called `b2`
ep1cman commented 1 year ago

in the previous comment your remote was called backblaze-b2 but this error is showing just b2. Maybe the - is causing problems. The quotes around 'false' is also odd, I will try to replicate

ep1cman commented 1 year ago

I can replicate the issue with quotes, but this is a known issue with docker-compose: https://github.com/docker/compose/issues/2854, basically docker-composes passes the string as 'false' which is not a valid bolean value due to the quotes. You shouldnt need quotes at all from my understanding, or quote the whole env variable e.g.

      - 'UFP_SSL_VERIFY=false'
Smith8154 commented 1 year ago

Yes, it would appear that the quotes are not needed for any of the values. I removed the dash and just called it b2, but that doesn't work either. It seems like no matter what I call it, it's not reading the value.

ep1cman commented 1 year ago

could you try running the following to see if your config file is getting properly loaded:

docker run \
  -v /path/to/your/config:/config/rclone.conf \
  ghcr.io/ep1cman/unifi-protect-backup:dev \
  rclone listremotes

if you ignore the s6-rc log entries, there should somewhere be a list of the remotes it has configured, for example I see:

[ls.io-init] done.
s6-rc: info: service 99-ci-service-check successfully started
local:
onedrive:
s6-rc: info: service 99-ci-service-check: stopping
Smith8154 commented 1 year ago
s6-rc: info: service 99-ci-service-check: starting
[ls.io-init] done.
s6-rc: info: service 99-ci-service-check successfully started
2022/08/21 18:05:53 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
Smith8154 commented 1 year ago

I did have an issue generating the config file, that appears to be a bug with rclone. If the files doesn't already exist, it was failing to create one and write it. I had to manually create a blank rclone.conf file, then run the command to generate the file before it would update the file with the correct data. Not sure if this would be anything permissions related due to that.

ep1cman commented 1 year ago

Hmm looks like the env variable for changing the rclone config file location isnt working as it was before. I will investigate further and get back to you

ep1cman commented 1 year ago

Aaaah, the readme has an incorrect rclone.conf path, try using /config/rclone/rclone.conf instead:

docker run \
    --rm \
    -v /path/to/rclone.conf:/config/rclone/rclone.conf \
    -e RCLONE_CONFIG='/config/rclone/rclone.conf' \
    --entrypoint rclone \
    ghcr.io/ep1cman/unifi-protect-backup:dev \
    listremotes
Smith8154 commented 1 year ago

Now it just outputs the below line and then exists. I assume that is what we want.

remote:
ep1cman commented 1 year ago

Ok good this means the config is being detected and I assume your remote is now called remote? I do wonder how anyone else got this working previously 😕.

Does this work for you now?

  unifi-backup:
    container_name: unifi-backup
    image: ghcr.io/ep1cman/unifi-protect-backup:dev
    environment:
      - UFP_USERNAME=my-username
      - UFP_PASSWORD=my-password
      - UFP_ADDRESS=my-ip
      - UFP_SSL_VERIFY=false
      - RCLONE_DESTINATION=remote:/unifi_protect_backup
      - RETENTION=14
    volumes:
      - '/mnt/docker/unifi-protect-backup/data:/data'
      - '/mnt/docker/unifi-protect-backup/rclone.conf:/config/rclone/rclone.conf'
Smith8154 commented 1 year ago

That gets it to start, but it still says my retention is 7 days.

Edit: forgot to answer your question, yes I called it remote because I thought it may just be looking for it to be called remote.

ep1cman commented 1 year ago

ah, you used the wrong env var name, should be: RCLONE_RETENTION, I was just copy and pasting yours

Smith8154 commented 1 year ago

Ah yes, the classic ID10T error. After updating that, everything is working properly! Thank for for your help and the fast turnaround on the ARM build! I will continue to watch the logs for the next few days and open a new issue if I come across any problems.

ep1cman commented 1 year ago

Thanks for helping test it!

ep1cman commented 1 year ago

Had to tweak the CI some, but there is now a new release with and arm64 container. You can go back to using :latest and all release from now on will come with an arm64 container.