Closed fedeD84 closed 1 month ago
I can confirm that.
try the old version
I’ve try to update after have the error the first time
I can confirm error
The site api has been updated. The Storytel support has to be reimplemented.
The site api has been updated. The Storytel support has to be reimplemented.
Is there anything we can do to help you reimplement it?
The site api has been updated. The Storytel support has to be reimplemented.
Is there anything we can do to help you reimplement it?
I did the last rewrite but currently don't have any active storytel subscription. If someone can share his login credentials I can have a look on what needs to be adjusted.
The site api has been updated. The Storytel support has to be reimplemented.
Is there anything we can do to help you reimplement it?
I did the last rewrite but currently don't have any active storytel subscription. If someone can share his login credentials I can have a look on what needs to be adjusted.
Hi, i cant provide credential, how can i send it to you?
You should be able to see my email in my profile now.
Mail send
Got them, thank you.
API didn't change but it seems like they added cloudflare as proxy and it's detecting and blocking (CAPTCHA page) requests made by python/requests. Detection seems to be more sophisticated (suspecting TLS fingerprinting, header order checking or something simmilar). IP Reputation seems to be relevant too (blocked from my server, working fine when proxing it via my Desktop). Not sure how well the current cloudflare solver proxies are working, I'll try to fix it without having to use one.
Got them, thank you.
API didn't change but it seems like they added cloudflare as proxy and it's detecting and blocking (CAPTCHA page) requests made by python/requests. Detection seems to be more sophisticated (suspecting TLS fingerprinting, header order checking or something simmilar). IP Reputation seems to be relevant too (blocked from my server, working fine when proxing it via my Desktop). Not sure how well the current cloudflare solver proxies are working, I'll try to fix it without having to use one.
Thanks a lot, we wait for your update and if needed make some tests
You can try the following change to make sure you've the same problem as me (blocked by cloudflare). https://github.com/jo1gi/audiobook-dl/pull/108/files It probably won't fix the blocking, it's just giving a better error message.
You can try the following change to make sure you've the same problem as me (blocked by cloudflare). https://github.com/jo1gi/audiobook-dl/pull/108/files It probably won't fix the blocking, it's just giving a better error message.
Hi I tried your code and yes i got nicer error message :-) I will se if I can help
No problem actually for me
I belive I tracked down the problem. It's indeed TLS fingerprinting related. The urllib3 default SSL context will set the SSL_OP_TLSEXT_PADDING option which will result in a padding extension in the ClientHello in many OpenSSL configurations. When unsetting the option (no padding extension) it seems to work fine.
I'll update the code tomorrow.
I've updated the storytell_cf branch. Please test if it's working for you. If you're also using other sources, please test them too (disabled the padding extension for all HTTPS requests).
I will test it later to night
I got this error:
requests.exceptions.SSLError: HTTPSConnectionPool(host='www.storytel.com', port=443): Max retries exceeded with url: /api/login.action?m=1&token=guestsv&userid=-1&version=24.22&terminal=android&locale=sv&deviceId=995f2562-0e44-4410-b1b9-8d08261f33c4&kidsMode=false (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))
not use https.
This version is working perfectly for me, thanks!
@nianhbg the change should have no impact on the certificate validation. If you just use the master version you should get the same error.
Ok thanks I got it working :-D
I've updated the storytell_cf branch. Please test if it's working for you. If you're also using other sources, please test them too (disabled the padding extension for all HTTPS requests).
im sorry if im stupid, but how do i install that branch?
Clone the repo and use git checkout storytell_cf
to switch to the branch.
i have gotten that far, but python3 setup.py install on root does nothing
i have gotten that far, but python3 setup.py install on root does nothing
Remove the previous installation with pip uninstall audiobook-dl
Then
pip install "git+https://github.com/jo1gi/audiobook-dl.git@storytell_cf"
Few bugs I needed to fix (Windows): 1.
pip uninstall audiobook-dl
pip install git+https://github.com/jo1gi/audiobook-dl.git@storytell_cf
2. https://github.com/jo1gi/audiobook-dl/blob/storytell_cf/audiobookdl/output/output.py#L113 not working on windows, changed to 256.
3. https://github.com/jo1gi/audiobook-dl/blob/storytell_cf/audiobookdl/sources/source/__init__.py#L180
ssl_context.check_hostname = False
4. https://github.com/jo1gi/audiobook-dl/blob/storytell_cf/audiobookdl/sources/source/__init__.py#L188
session.verify = False
@thewh1teagle thank you for the feedback.
What python and windows version are you running?
ssl_context.check_hostname = False
and session.verify = False
should not be necessary and disables HTTPS security.
Any chance that you have some fancy security software installed which intercepts web traffic?
I get the same SSL errors as https://github.com/jo1gi/audiobook-dl/issues/106#issuecomment-2168742753 and the ones I guess https://github.com/jo1gi/audiobook-dl/issues/106#issuecomment-2191463839 tries to avoid with turning off verifications.
requests changed how certificates are loaded in 2.32.3. Not clear if it is a bug or not. https://github.com/psf/requests/issues/6730 If using requests 2.32.2 everything works.
With 2.32.3 it worked for me (Mac OS Sonoma, pyenv, 3.11.9) after adding either a
ssl_context.load_default_certs()
or
import certifi
ssl_context.load_verify_locations(certifi.where())
or something similar in the create_session function
Thank you @currently-off-my-rocker
You're absolutly right.
I looked into this, and requests introduced a regression with the 2.32.0 release which breaks the usage of custom SSLContext's (Which we need for the cloudflare). It's fixed in 2.32.3, but breaks certifiate loading. I added the ssl_context.load_default_certs()
workaround in #113.
Also fixed Windows support via #114 Please test the current master branch or the https://github.com/jo1gi/audiobook-dl/releases/tag/v0.7.4 pre release.
i installed it using pip install "git+https://github.com/jo1gi/audiobook-dl.git".
audiobook-dl https://www.storytel.com/nl/nl/books/book
Traceback (most recent call last):
File "C:\Python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Python39\Scripts\audiobook-dl.exe__main.py", line 4, in
@zaourzag please try again, made some adjustments for older python versions
@zaourzag please try again, made some adjustments for older python versions
audiobook-dl https://www.storytel.com/nl/nl/books/book
Finding compatible source
Traceback (most recent call last):
File "C:\Python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Python39\Scripts\audiobook-dl.exe__main.py", line 7, in
perhaps i should just use a newer python version
Please test the current master branch or the https://github.com/jo1gi/audiobook-dl/releases/tag/v0.7.4 pre release.
tag v0.7.4 works on my setup.
perhaps i should just use a newer python version
Yes, that would help, can you check which exact python and urlib3 version you're running currently?
im running python v3.9.13 and urllib3 1.26.19. edit it was the urllib3 version. i did pip install --upgrade urllib3 and it fixed it
is it maybe an idea to add an optional flareresolver "proxy"? it essentially bypasses cloudflare blocks.
is it maybe an idea to add an optional flareresolver "proxy"? it essentially bypasses cloudflare blocks.
If it ever becomes mandatory, that's probably the logical solution. For now it shouldn't be necessary.
so an update, I have been using my laptop. it now throws a blocked by cloudflare error. so i moved to my desktop. it can download just fine. both devices are on the same network and on python 3.9. so i dont know the trickery cloudflare employs here
Made urrlib3 version > 2 mandatory on the latest master branch. @zaourzag maybe you can check which packets are different by running "pip freeze" in your two virtaulenvs.
Made urrlib3 version > 2 mandatory on the latest master branch. @zaourzag maybe you can check which packets are different by running "pip freeze" in your two virtaulenvs.
i dont understand what this would help. it worked on both devices one is now just blocked by cloudflare for whatever reason
Made urrlib3 version > 2 mandatory on the latest master branch. @zaourzag maybe you can check which packets are different by running "pip freeze" in your two virtaulenvs.
i dont understand what this would help. it worked on both devices one is now just blocked by cloudflare for whatever reason
Just to double check that both are really running the same package versions. Maybe your laptop is running a buggy version of requests.
Just to double check that both are really running the same package versions. Maybe your laptop is running a buggy version of requests.
my laptop has requests 2.32.1 and my desktop has 2.31.0. dont think requests made any breaking changes in that time
Just to double check that both are really running the same package versions. Maybe your laptop is running a buggy version of requests.
my laptop has requests 2.32.1 and my desktop has 2.31.0. dont think requests made any breaking changes in that time
i just installed 2.31.0 on my laptop and it works again. what sorcery is this i wonder
2.32.0/2.32.1/2.32.2 are all buggy. Try 2.32.3 (latest), it contains a bug fix which is important for the cloudflare check. It introduced another bug, but the master branch contains a workaround for that bug :)
2.32.0/2.32.1/2.32.2 are all buggy. Try 2.32.3 (latest), it contains a bug fix which is important for the cloudflare check. It introduced another bug, but the master branch contains a workaround for that bug :)
will do, still dont understand how 2.32.1 worked for a bit then didnt and 2.31.0 just did. edit: tried it, works with 2.32.3
also the issue with urllib3 was that it was not getting installed by this project (not in the pyproject.toml file) so glad you fixed that one. took me a while to figure out lol
Until 3-4 days ago i haven't got problem with the download from storytel. In the last day every audiobook i'm trying to download get me: ERROR: User not authorized User and password are correct, i've recreate the cookie files but nothing change.