search

jeffreydwalter / arlo

Python module for interacting with Netgear's Arlo camera system.
Apache License 2.0
517 stars 123 forks source link

Arlo Limits? #83

Closed ariaceleste closed 6 years ago

ariaceleste commented 6 years ago

Hi Guys,

To begin, I must commend you on your fantastic work here. This was easy to install and get running on my Windows machine using Cygwin for a semi-new developer and Linux hack. Kudos to all!

I'm unsure if it's an SSL error or an Arlo server limitation, but I set up a scheduled task to pull my videos every hour. At first it was working perfectly, but then I started to run into HTTPS Connection Pool errors and other connection errors (I think 60034 was one). Using a VPN to hit it from a different IP Address worked once or twice, but then stopped working.

I'm currently stuck with about 500 videos in my backlog and can't get a complete bulk download. I'm running Python 3.7, made sure TLS 1.2 is enabled, and installed a few weeks ago so I should have the most updated code.

Can you please advise? Thank you :)

jeffreydwalter commented 6 years ago

@ariaceleste thank you for your complements. Can you please edit your original comment and answer all of the questions that were there in the template for you to answer? Please see https://github.com/jeffreydwalter/arlo/wiki/Issue-Guidelines-and-Policies

Thank you.

ariaceleste commented 6 years ago

@jeffreydwalter My sincere apologies. I'm not currently on the computer where I'm running the code, but I am making a note to attach that information tonight when I return to my computer so you have a complete record.

I believe I have this issue resolved. I'll file my details here in case anyone has similar issues and peer review.

As stated, I had a scheduled task (using Windows) to run arlo.py every hour. It worked fine for a while but then I started getting HTTPS Connection Pool errors and eventually I'd get 10054 connection issues.

Through some harried research as my video backlog accrued, I resolved the HTTPS Connection Pool error by assuring exit() was performed in Python. Upon next load, that error resolved.

Shortly thereafter, I then came across the 10054 error, which I interpreted as Arlo denying me as a potential DDoS attack. I figured at a minimum Arlo's AWS servers were using Shield, so I did a few things:

(1) I created a new service account at Arlo that I'm using solely for your code (2) I assured to use a VPN so I could switch up my IP address (3) I resolved to only hit the Arlo servers once a day and I monitor download speed

I've successfully downloaded somewhere between 300 to 500 videos each day using this method for the past few days. I share here both to check my testing work as well as provide you more information as it may help you to further improve the product.

If I find some time (again, I'm a new developer so coding takes me a while), I'm going to try and make a few modifications:

(1) Direct save from Arlo S3 to Dropbox (2) Work in a try:catch statement so I can get an email notification if it fails (3) Encrypt the login credentials, just in case :)

After that I hope to either install the code on a Raspberry Pi or set up an AWS Lambda Function so I can auto-download. :)

Thanks again for all your help guys. This is my first "project" on Github and it's great to be a part of such a welcoming community.

jeffreydwalter commented 6 years ago

@ariaceleste no problem, happy to help. If you have it working, no need to fill out the questionnaire.

I don't have anything that hits Arlo's servers (or AWS), so I've never encountered that code (10054). Seems plausible that it's a quota limit.

I'd definitely appreciate your follow-up info.

Thanks!

ariaceleste commented 6 years ago

I'm closing issue and will keep in touch as I progress through this.

Thanks for the prompt response @jeffreydwalter and keep up the good work guys!

ariaceleste commented 6 years ago

@jeffreydwalter

ariaceleste commented 6 years ago

@jeffreydwalter One final question: If you don’t pull the videos from Arlo, from where do they come?

Apologies for the stupid question. :)

jeffreydwalter commented 6 years ago

@ariaceleste no such thing as a stupid question. :) The videos are recorded by your Arlo camera, sent to Netgear's Amazon AWS account.

To get the videos with this library, you would call the GetLibrary() method to get the list of available videos (see https://github.com/jeffreydwalter/arlo/blob/master/Arlo.py#L850). If you look at the comment at the top of that function, you will see the JSON that is returned by that API call. It includes the presignedContentURL field (see https://github.com/jeffreydwalter/arlo/blob/master/Arlo.py#L834). This field contains a temporary URL to the video in AWS.

Hope that makes sense.

Cheers, Jeff