search

jo1gi / audiobook-dl

Audiobook CLI downloader
GNU General Public License v3.0
252 stars 43 forks source link

Chirp: Error while making request to server (audiobookdl.exceptions.RequestError) #65

Open Andronian42 opened 1 year ago

Andronian42 commented 1 year ago

Installation method: pip (pip install audiobook-dl)

Version: python 3.12.0 (tags/v3.12.0:0fb18b0, Oct 2 2023, 13:03:39) [MSC v.1935 64 bit (AMD64)] audiobook-dl 0.7.0

Describe the bug Attempt at downloading from Chirp fails after getting a response from the server. Cookie txt file was just updated by me (using this extension on Firefox) so should be up-to-date. I've tried multiple different audiobooks from the site, including ones that I've downloaded through audiobook-dl before, and have gotten the same error.

Command output

PS Z:\drive2\Audiobook\.download> audiobook-dl -d -c Z:\cookies-chirpbooks-com.txt --output-format m4b https://www.chirpbooks.com/player/12327174
DEBUG audiobook-dl 0.7.0
DEBUG python 3.12.0 (tags/v3.12.0:0fb18b0, Oct  2 2023, 13:03:39) [MSC v.1935 64 bit (AMD64)]
 INFO Finding compatible source
 INFO Authenticating with chirp
DEBUG Loading cookies from 'Z:\cookies-chirpbooks-com.txt'
DEBUG Downloading result of https://www.chirpbooks.com/player/12327174
DEBUG Failed to download data from: https://www.chirpbooks.com/player/12327174
Response:
b'<!DOCTYPE html><html lang="en-US"><head><title>Just a moment...</title><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"><meta http-equiv="X-UA-Compatible"
content="IE=Edge"><meta name="robots" content="noindex,nofollow"><meta name="viewport" content="width=device-width,initial-scale=1"><link href="/cdn-cgi/styles/challenges.css"
rel="stylesheet"><meta http-equiv="refresh" content="375"></head><body class="no-js"><div class="main-wrapper" role="main"><div class="main-content"><noscript><div
id="challenge-error-title"><div class="h2"><span class="icon-wrapper"><div class="heading-icon warning-icon"></div></span><span id="challenge-error-text">Enable JavaScript and cookies to
continue</span></div></div></noscript></div></div><script>(function(){window._cf_chl_opt={cvId: \'2\',cZone: "www.chirpbooks.com",cType: \'managed\',cNounce: \'29360\',cRay:
\'823ba11a6e77e259\',cHash: \'398fd114b97a7b4\',cUPMDTk: "\\/player\\/12327174?__cf_chl_tk=NFCgzh4GFz9gDkHgzKHoF5t36XUnEFzHZzT5oqsVoHM-1699591351-0-gaNycGzNDKU",cFPWv: \'g\',cTTimeMs:
\'1000\',cMTimeMs: \'375000\',cTplV: 5,cTplB: \'cf\',cK: "visitor-time",fa: "\\/player\\/12327174?__cf_chl_f_tk=NFCgzh4GFz9gDkHgzKHoF5t36XUnEFzHZzT5oqsVoHM-1699591351-0-gaNycGzNDKU",md:
"Gi5weDVHK7PeXhrwqAPO088qDdEHe733F7g85nEg.gc-1699591351-0-ASbuw_avuHFJEY-iiWJtWKA3sXUPrIiL9qAs8JHwDVAi4xTSO5EgzEWcLdbTWhg3i-nrWUjYTK1XJaP87Jt_jFWE0d4iWyKoznziSOMERqKs1NllptIu2yVJPF1qAewd1rX
6oVBPgQEe67bvXEh520L1KJKLorq-gRav_5NfGJHniGtPCOcN4H6XUpfvA2CJKOVUke5KVAo_270i-7Fqn5tFVlSsJXcPr4zq_6mS-OBJKQsbQNSV8THJ0gm3Ca3bMmy0f21Cc5LaxA2O9NxPKK8fdwU9a68u2Rsd0UIq_ui32OZ_4Uoc2pkjsAW5mqXC
taAOgq_X1lejEwtLR8rEm38g5MiqGzJ4RlnkJLFAwJsqNtbOuXIY7lns_SRFdt42cK_qwZvhzml8wynlikKRdzUAFl3J4jA_t01Dye2dBiNI1BTH1qcZ30-pI6KXaH95OOWdyHYaFHk_vy3W4lCUe6aZtfbhkb4ziy-6WgtdZRGeSeGLG7mAvYpyFBhoP
2Vq1SBz7PTVX4plCxDxRg2FEkT_OMwpgTf9xEWUroH7JNsuHw2oJ8slSLcohoYQD0nQVjrxtNMeyObWZ-uohiIpDtRJXnhhyEM7bV3qnHzfGCvjCddmR-DIOPCblvjKPBjYDQ4XgBZuTKYPVL4gHMxpiJqL-y2DQ2Y9TuLGEqQDNJcLSb9fiydsvQd5jc
1ERCvd-nIjorRlQlxv_RrtSVCNZlRaDqSYfveQhLm-MaLE0e1fBJac0Ucx03EU3LmYiMq1b6sgNrqb1Vn-ctwPACbgCWX-sTTH1r7BWN_E5g8owxciGFYv760zOqkwzed_uVU_U4RfuxbosNo015V1GRQO6ESsVSfRPdeTudOC2x6ohQm3N24oEv94Zpu
FXbATaKkgXdZtOMcgGvIODougl90T23qbDEM9K3y8gVSdvq86iTy74hpVNivMM4AAB4GGu3GwdYqXTI4Oow1fxaES8-OYTFptM-D2MWCaTgAa-o271DKRPD3BBrPB52mhNcxweHLa6mJLuOkqCJCDq5YXP2qjjhkv0EfCYQCbdjcMvakeNcWJGszqLv7T
kdk_ViPhRw7RtojW_B8_-wOS-NlxMJbDG2IFV23Z5L33C5phVD6JvXBnq2VMRSBtdbUNv2WYgsMPPS4cO4kCsFrYjps7yyeuiEBF-pKM1MjF9x9YDFcOk2fNmMRsr3q1Ttox0xfDl8TQMaJjXVFJ6b3CJR78Fx2Vl-TZpjPORDyIlGDtxhqx9kseh-sCO
vJwQEYHrZDrxmRUThRDzxWvfuyH7P7lI79d5KpdlA1dMef0NtsDmobBAR8zJHXbqDTuj31e8I3o4shSrSbLbADDaEXu3oxSg2n5LIJjwv3-USx_JLN65gU6_38CeGzz-5_28uOzKkICMCbq7vwKcCd0Rf9gjzWbIsfbCmxoDmhFPw5jLVxRI1ZsqFAsRL
ako-_cBVw-RUhR-dkGWX7NeEKHXhDLWNz4UanAu2dzOB-Bb-p1gwuaZDmxkd6Mls9u7oZcNAsD6bzEXsvPwLR3bd4xncNEHaZ2g9W5EmeaeM3Ynu9C2S6r_5Dw7lv5JO3deY_yOXdjhFKluKImPWwqVoE-552e6pVQa8Y80AN3b-JIVaYnR798qknHsIR
0skfWN7jsUOOUq-gBxNVu4K6UYuES6sX_hQZmN_tGWb18XCmS9dPXpoxt835eY8mQZNDJHgDxs8XlyfEs-tBqbBdDa0y2lrgOXbvjRZhGDnitGVA9rnPpAuM4W8yDrXJokpb1T2NAtQ7F45OLYTsZDhq9bHH50PEX-m3LfvEOlXi5T1nMtCIVVXK7S7kd
Cdm-QaOVU2bMhj3tlzAkKVDtVTZq_JPuv1vsMtDpyQcWndwSfPsrHiu_b0-QGLt6kkLWNRNQEATAb-1htCgjcLJYaNXxg4nRpH_e_GoXfwj8SJFoQwt1VpIvfCIBg3LcvbL0SlJcHPlqqVH-KOEUPcJ5OsklIw_MD2Gmy3Kd0NqY_Sb9aYCT3TaPetEq9
Oqe028u_VSq8R4a1UZl34-BXORJen0ogKDmHu1-hXym72yeu4P3tEnv2iFGTGtrYAFOB3mDVf1pcNt25lyFfdssLGrInZgI_MjgFf0-0OpYE7UCBP3nem77-KHGGU9dEay1cIuiLCIPPH9-MyRzL0k22smmmw_x5ELjtGnSvrH5aGGJ4DPLUVXKSusFT5
_LL3gSa_sOoOT6Hprb_p9HUhVwezniVbJG6md7Y2JDmAjO2IOoJ3Ln_hgElPjigjAh4fj1uZQPEHDpzq5_ZLsZdhQDUB3ffr-MzC_9grtr38rgL4pDykHJjC_eiM-jzp_3LWq41niC1xJX2pTbzAWS2GW3XL5BCU12qz4_jigbVmhjsf-2ralWwYW_T28
Iftjwr6VY4eC1hzbeBx80dHLiWnNnux1SzRNdD85tD3R3D-e2UDgKna4x-eve_XV6XAQMBfZ5VxQMKOoc_RejefEe9eB6OXH8DLtDB4Fjf19Cp8T-JjvT2IzYMcq7at0Gsr1oTWGYPRmHz6tr7jzyclr676jkpF47mmF0RBxK8YMGDdG9TTELv1_vVYBE
fEbMfxVIziqsfk8AUAHXJrSdGNy8mg1X0ydWsMa0zaXH6GM0zmt_R6f_faijEzFoMI5gEaR26FX2vdva3SfX6ZA113DgHQvSA-_SEQe1YFmNBenMD-t1gPkI8-CQ1B1-rwWBwSLrFLaXay_KyWLif08OCeTNZsdKcNAdDfUjcwO3Tbje2hxX-zpTew92S
KPIGDJDO9K-XxZtP1cMtQyakar8TNqv2pU7sIIzRWm1TgGV9agbMSwQQRVsu0yIvDx2Yh3P84kG_8Gkn3zs-y4zST-yq4phtXTN2MhDR1xVo-te0pXnEaAcYM6Dn1A1F2dyJd5u2KWbnx5QrQbQ5V1asPsl8CcUUEdNvJ6S8JtBMb76T9GsgPwtux25Ki
m68ijnSmKm6TarqlAQspbTDBUHbV88mPxZDXzzncW6X_i15Xq4jhiYb-xL8X1Y3zV7YICneHHrp0YoZ_g3gpv3qefq-ARkB7_n7VcGs2ZKr3gPomcbSuqBFYawv0pe-ap9tElpGvVaoLqekyvhCAbg-JRpoCjlGxiVyoBMprH2BjjrgO3ZdTQv1o5raMB
OEXrfIhmRqVhNGZFurbTKCoRc",cRq: {ru: \'aHR0cHM6Ly93d3cuY2hpcnBib29rcy5jb20vcGxheWVyLzEyMzI3MTc0\',ra: \'cHl0aG9uLXJlcXVlc3RzLzIuMzEuMA==\',rm: \'R0VU\',d:
\'1qTB3QP5qCksiwGdrBtqLwvBeWPGcqpjv3Ou8vMyS8X6aJZKdhhuWjkPbwIzQw6ZwwzIfG7BYZaxpWJ8fLsLMHGxWLSJm1jYhUgQiYF85drchagiFG3Hc6BaFuJVakTzGl/4e17tfCwZcmy3KNAuno7vRRUmh6RnycQHntVRQgGfzVffapmrxtu/pZy
CLhsZxtMpmDkbFczkHzDKOkNEX/P0XVPnc6TecY778x4on9jzKH2tuWS8r6ces+50raU3oyRrPiuz4MS9ywXR6Q+zjD/MKyME+rZWxiHVR0VgAeOvQGQs3n6R7uhS1UOWQ9Z0Bto0IWJrA5ixZM6GXbSN1BJBqufmP+ErzGv2MLHY8XW/amBvEsiO65oa
PAqyXUCa2R0XgKXkFc4RPTDtPJpzScHKjStJMR0wNgcyaLTOWixVPTPQiaETaw6z9U1Uu4b7k+TQnEyNbPiG1VT8bR4keOYO6jlu77+GlkmEAZ0wyziEWx5Uyk6fXUwax0daC84bTZwwvmjr86dVFbfS9d12MH9A8INfog0F17140f/+Ig9aBJBfxzxBG
hTpLZslDfg7IsD8Z0W/41XiJgsfimoAkYvgGoDheNI0V4ma89xjo3M=\',t: \'MTY5OTU5MTM1MS40MjkwMDA=\',cT: Math.floor(Date.now() / 1000),m: \'8utByCbKS+XoIfCd3iu9E8t+8GjmeavC8iFoZLITkuc=\',i1:
\'lz2Q/yV9NXb3metB4+FG+w==\',i2: \'YlvsuCJeuc2psEV/hBrpZg==\',zh: \'Y2bLFLW8//0CywAGfr1u8fJ5lqFIwPuusDg+If3CYJY=\',uh: \'YE9XOpG5TeHmhA1zfs5mxC8CrRZzq2a/+r+OU7dliYQ=\',hh:
\'sznXXJzkrNMuaryFxrTQ0GkMcBJhn4v6vHXSgGTxOAY=\',}};var cpo = document.createElement(\'script\');cpo.src =
\'/cdn-cgi/challenge-platform/h/g/orchestrate/chl_page/v1?ray=823ba11a6e77e259\';window._cf_chl_opt.cOgUHash = location.hash === \'\' && location.href.indexOf(\'#\') !== -1 ? \'#\' :
location.hash;window._cf_chl_opt.cOgUQuery = location.search === \'\' && location.href.slice(0, location.href.length - window._cf_chl_opt.cOgUHash.length).indexOf(\'?\') !== -1 ? \'?\' :
location.search;if (window.history && window.history.replaceState) {var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;history.replaceState(null, null,
"\\/player\\/12327174?__cf_chl_rt_tk=NFCgzh4GFz9gDkHgzKHoF5t36XUnEFzHZzT5oqsVoHM-1699591351-0-gaNycGzNDKU" + window._cf_chl_opt.cOgUHash);cpo.onload = function() {history.replaceState(null,
null, ogU);}}document.getElementsByTagName(\'head\')[0].appendChild(cpo);}());</script></body></html>'
ERROR: Error while making request to server
Traceback (most recent call last):
  File "C:\Users\[redacted]\AppData\Roaming\Python\Python312\site-packages\audiobookdl\__main__.py", line 32, in main
    process_url(url, options, config)
  File "C:\Users\[redacted]\AppData\Roaming\Python\Python312\site-packages\audiobookdl\__main__.py", line 54, in process_url
    result = source.download(url)
             ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\[redacted]\AppData\Roaming\Python\Python312\site-packages\audiobookdl\sources\chirp.py", line 21, in download
    book_id = int(self.find_elem_in_page(url, "div.user-audiobook", "data-audiobook-id"))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\[redacted]\AppData\Roaming\Python\Python312\site-packages\audiobookdl\sources\source\__init__.py", line 116, in find_elem_in_page
    results = self.find_elems_in_page(url, selector, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\[redacted]\AppData\Roaming\Python\Python312\site-packages\audiobookdl\sources\source\__init__.py", line 132, in find_elems_in_page
    page: bytes = self._get_page(url, **kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\[redacted]\AppData\Roaming\Python\Python312\site-packages\audiobookdl\sources\source\__init__.py", line 103, in _get_page
    resp = self.get(url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\[redacted]\AppData\Roaming\Python\Python312\site-packages\audiobookdl\sources\source\networking.py", line 33, in get
    raise exceptions.RequestError
audiobookdl.exceptions.RequestError
PS Z:\drive2\Audiobook\.download>
jo1gi commented 1 year ago

I think Chirp has introduced bot detection. I will see what I can do about it.

joshaconnor commented 1 year ago

same error here. i noticed a hidden recaptcha element in the html of the player, but idk if that was there before or not. wanted to comment to boost the issue too, but hopefully the captcha thing helps

joshaconnor commented 1 year ago

follow-up: after looking into it a bit more, i think they are using cloudflare protection, so integrating something like flaresolverr or another solution might solve this.

toonvank commented 5 months ago

same issue here