search

jeffreydwalter / arlo

Python module for interacting with Netgear's Arlo camera system.
Apache License 2.0
517 stars 124 forks source link

fix cloudflare 403 with curl_cffi #205

Open bjia56 opened 11 months ago

bjia56 commented 11 months ago

Cloudflare frequently returns a 403 on the Arlo login, which to the best of my understanding is decided on client heuristics, i.e. if a client is determined with some confidence to not be a browser, the request is blocked. One of the heuristics used is the TLS fingerprint of the client. The curl_cffi project aims to enable Python scripts to bypass this TLS fingerprinting check through creating TLS handshakes that are identical to a real Chrome browser. This is accomplished through the use of the curl-impersonate project, which compiles curl with the exact TLS/SSL libraries used by Chrome and Firefox.

We have seen reasonable success in using this option over cloudscraper in the Scrypted home automation project. A caveat is that curl_cffi exposes a requests-like API, but does not implement the full API; therefore, curl_cffi is only used for the initial login, and subsequent requests fall back on requests so sseclient can continue working. Arlo appears to be happy with just the token in the header, and performs no additional TLS fingerprinting checks after login succeeds.

Note that this is not tested comprehensively, since I do not use this version of the arlo library.

Fixes #204

Johnathan-Aretos commented 11 months ago

Can confirm that this works as expected. Before this I had problem with authenticating.

ahass-thedev commented 11 months ago

Works as expected - Was not able to authenticate before

scuc commented 11 months ago

I tried to run this fix, but it throws an error - see below. I can confirm that /usr/lib/libSystem.B.dylib is present on my system. I have tried to run this code from both pipenv and just a python3.10 local install. but I get the same error each time. only part of the traceback that stands out to me is - "built for Mac OS X 11.0" i'm running on Mojave 10.14.6 - have not been able to test on Mac OS 11 system.

Traceback (most recent call last):
  File "/Users/<user>/_Github/arlo-fix/arlo-cloudflarefix/main.py", line 4, in <module>
    from arlo import Arlo
  File "/Users/<user>/_Github/arlo-fix/arlo-cloudflarefix/arlo.py", line 27, in <module>
    from request import Request
  File "/Users/<user>/_Github/arlo-fix/arlo-cloudflarefix/request.py", line 19, in <module>
    from curl_cffi import requests as curl_cffi_requests
  File "/Users/<user>/.local/share/virtualenvs/arlo-cloudflarefix-ZHEWMQYK/lib/python3.10/site-packages/curl_cffi/__init__.py", line 4, in <module>
    from ._wrapper import ffi, lib  # type: ignore
ImportError: dlopen(/Users/<user>/.local/share/virtualenvs/arlo-cloudflarefix-ZHEWMQYK/lib/python3.10/site-packages/curl_cffi/_wrapper.abi3.so, 2): Symbol not found: ___darwin_check_fd_set_overflow
  Referenced from: /Users/<user>/.local/share/virtualenvs/arlo-cloudflarefix-ZHEWMQYK/lib/python3.10/site-packages/curl_cffi/.dylibs/libcurl-impersonate-chrome.4.dylib (which was built for Mac OS X 11.0)
  Expected in: /usr/lib/libSystem.B.dylib
 in /Users/<user>/.local/share/virtualenvs/arlo-cloudflarefix-ZHEWMQYK/lib/python3.10/site-packages/curl_cffi/.dylibs/libcurl-impersonate-chrome.4.dylib
bjia56 commented 11 months ago

The published curl_cffi package may have been built on a newer MacOS version, which unfortunately means it is not guaranteed to be backwards compatible to run on older versions