AttributeError: 'NoneType' object has no attribute 'minute'

[tlalexander]

Hello,

Thank you for making this software!

I have a UNVR I would like to back up footage from. It seems like I need to follow "UDM based" setup?

I start the proxy and then run the archiver.

docker run --network=unifi-protect-video-downloader_default --volume /home/taylor/Videos:/downloads unifitoolbox/protect-archiver download --address="unifi-udm-api-proxy" --username xxxxxxx --password xxxxxxx --cameras xxxxxxx /downloads
Getting camera list
Successfully authenticated as user xxxxxxx
Successfully retrieved data from /api/cameras
Cameras found:
<REMOVED>
Downloading video files between None and None from 'https://unifi-udm-api-proxy:7443/api/video/export' for camera MyCamera

Downloading footage for camera 'MyCamera' (xxxxxxx)
Traceback (most recent call last):
  File "/usr/local/bin/protect-archiver", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/protect_archiver/cli/__init__.py", line 20, in main
    cli.main()
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/protect_archiver/cli/download.py", line 199, in download
    client.download_footage(start, end, camera)
  File "/usr/local/lib/python3.8/site-packages/protect_archiver/client.py", line 335, in download_footage
    for interval_start, interval_end in calculate_intervals(start, end):
  File "/usr/local/lib/python3.8/site-packages/protect_archiver/utils.py", line 46, in calculate_intervals
    start_diff_to_next_full_hour = diff_round_up_to_full_hour(start) - start
  File "/usr/local/lib/python3.8/site-packages/protect_archiver/utils.py", line 13, in diff_round_up_to_full_hour
    if date_time_object.minute != 0 or date_time_object.second != 0:
AttributeError: 'NoneType' object has no attribute 'minute'

I removed some PII, but nothing of consequence.

I originally downloaded the docker image from docker hub, but perhaps I can try building it myself with some modifications to the code for debugging.

Thank you!

[tlalexander]

And if I replace "download" with "sync" in the above command, I get the same error as in #36

[danielfernau]

Hey @tlalexander

Thanks for trying it out, and for reporting this issue!

Yes, currently there are a number of issues with both the Protect API connector as well as the project's code itself. Sadly, I don't have enough spare time right now to properly maintain the project or fix things as fast as I would like to – but I'll try my best to work through the pending issues and release updates whenever possible.

I've added build and development instructions today, you can find them here: https://github.com/unifi-toolbox/unifi-protect-video-downloader/blob/master/BUILD.md If you find a solution to this or have additional questions just let me know. Pull requests are also very welcome. Happy coding!

[tlalexander]

Okay I made a bit of headway.

I can add print statements to the python and run it and see whats going on in more detail.

It seems that various parts are working. It succeeds with auth and reading camera names, as we saw in my original post.

But when it tries to download footage, the response comes back with a 500 status code.

Here is the URL it tries to download, with my camera ID blanked out: https://unifi-udm-api-proxy:7443/api/video/export?&camera=XXXXXXXXXXXXXXXXXXX&start=1614902400000&end=1614905999000

So that URL returns a 500 error. Unfortunately I cannot test it in the browser as that unifi-udm-api-proxy address does not resolve. But I did find out what URL my browser uses when I download footage.

So lets say my UNVR is on 1.1.1.1 on my local network. Then if I go to the browser with a URL formatted like this, it will download the footage:

https://1.1.1.1/proxy/protect/api/video/export?&camera=XXXXXXXXXXXXXXXXXXX&start=1614902400000&end=1614905999000

I did try inserting that format in to the code, but it still gets a 500 error. Perhaps due to lack of auth, though maybe that would be a different error. Note the updated URL includes "proxy/protect/api".

Unfortunately I don't quite understand how the proxy works, so I'm a little confused how to proceed. Any thoughts?

[tlalexander]

Uh, wow okay I got something working! I can download footage now, but it's a bit of a hack.

Most everything is kept as default. I still use the proxy to connect and that connects and gets camera names and other metadata nicely.

But I changed the download URL to download directly from my UNVR using the second URL format above. In the python code I changed this line to the following. Say the IP address of my UNVR is 1.1.1.1, then I change the line to: address = f"https://1.1.1.1/proxy/protect/api/video/export?camera={camera.id}&channel=0&start={js_timestamp_range_start}&end={js_timestamp_range_end}"

Okay that's the first thing. But auth will fail if thats all you do. Then I went in to my UNVR, turned on the browser inspector in firefox, went to the "Network" tab, and initiated a download for a short section of video. Then I sorted by largest file in the view of network items, and I found that request. I clicked on it and noticed the cookie. (I actually copied it from "Request header" but messed up my screen shot. It's the same value though.)

So I copied that cookie value. Then you can add the cookie to the download request in the download_file function like this:

                my_cookie = dict(TOKEN="your_very_long_cookie_string_blah_blah_DMtYTc0MS0yZTcyNDVMS0yZTcyNDV_it_is_much_longer_than_this..")
                response = requests.get(
                    uri,
                    headers={"Authorization": "Bearer " + self.get_api_token()},
                    cookies=my_cookie,
                    verify=self.verify_ssl,
                    timeout=self.download_timeout,
                    stream=True,
                )

And then it works!

So there is some issue using the proxy to download video, but the other stuff that runs when you try to download video all seems to work. So this seems solvable. I looked at the code for the proxy but I think I've done enough hacking for now. Any thoughts? Thanks!

[danielfernau]

Thanks a lot for the amount of work you put into figuring out and documenting what causes these Status 500 errors, @tlalexander – a very thorough and clear summary!

A while back I started working on replacing the authentication part; branch danielfernau/v2.0.1/improve-client-authentication. Based on your posts it looks like the Protect system (in some cases?) rejects requests that only contain the Bearer Authorization headers but no TOKEN Cookie. I noticed in the past that the authorization needs to be refreshed every now and then – and there probably is some kind of connection between this behavior and the cookie.

In addition to that, it appears as if solving the problems with the authentication part of the tool might actually solve a bunch of the other open issues as well. I'm really looking forward to figuring this stuff out as soon as I have the time to do so, and will definitely use the things you've documented here along the way. Also, I'll attempt to update the tool in such a way that it no longer depends on the external API proxy and handles the communication internally instead. Less dependencies = less stuff to worry about.

I'll post updates here (as well as to the other issues) accordingly once I make progress and/or release an update. Hopefully I can start working a bit more on the project again in the next few weeks!

[danielfernau]

Hey @tlalexander

The first free weekend in months – I was finally able to implement the long-overdue changes in my app 😄

Release v2.0.1 is now available and should be fully compatible with the new UniFi OS authentication mechanism as well as the updated Protect API.

The UDM API Proxy is no longer required and everything works with one simple Docker command line at this point.

Thanks again for your help figuring out and documenting the API changes, and also for your patience! Happy testing – and let me know in case you encounter any issues with the new version.

[tlalexander]

Hey thats great! I used my hack to download a few important videos but I have really wanted this to be a more reliable system I can run all the time. I'm glad you did this! Note that my UNVR system is now telling me to update to some new interface they have released. 😭😭😭

But I will avoid the system update and give this a test!

[danielfernau]

Actually, the new UI isn't that bad. I quite like it, to be honest. Works fine on my system over here (UDM Pro) and makes the interaction with cameras, views and events a bit faster and easier.

The new network controller UI however, while awesome design-wise (in my opinion), has missing features and user experience flaws all over the place 😬 So I totally understand why you might not want to upgrade.

Either way – it's one version or another of UniFi OS, so the updated tool should work even if you're not using the latest firmware.

[las3r]

This issue is still the case when running the .whl from a non-docker context, I tried the 2.0.1 release as well as the current version (2.0.2) in git.

The script connects, gets a list of cameras with no problem, and then crashes with the exact same error message that @tlalexander mentioned. I'm running a Cloud Key Gen 2 plus (firmware version 2.1.7) with Unifi protect (1.18.0).

I haven't tried running w/ docker (I'm running workloads that prevent me from using hyper-v).

[danielfernau]

Thanks for the feedback @las3r ! In that case I'll keep the issue open and move it to the next milestone for further investigation. Which OS and Python version are you using? (it could be a platform-specific problem)

[las3r]

I'm using Windows Server 2019 Datacenter edition build 1809 (dec 2020). Python 3.9.0 (tags/v3.9.0:9cf6752, Oct 5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)] on win32

[danielfernau]

Thank you. I'll see what I can do.