scribblehub returns a 403 error when downloading stories

[kaidadragonfly]

$ fanficfare -d https://www.scribblehub.com/series/60207/twisted-destiny/
FFF: DEBUG: 2022-08-29 10:26:33,764: cli.py(230):     OS Version:macOS-12.5.1-x86_64-i386-64bit
FFF: DEBUG: 2022-08-29 10:26:33,765: cli.py(231): Python Version:3.9.13 (main, Aug 11 2022, 14:05:15)
[Clang 13.1.6 (clang-1316.0.21.2.5)]
FFF: DEBUG: 2022-08-29 10:26:33,765: cli.py(232):    FFF Version:4.15.0
FFF: DEBUG: 2022-08-29 10:26:33,789: configurable.py(1044): use_browser_cache:
FFF: DEBUG: 2022-08-29 10:26:33,790: configurable.py(1058): use_basic_cache:true
FFF: DEBUG: 2022-08-29 10:26:33,803: adapter_scribblehubcom.py(118): URL: https://www.scribblehub.com/series/60207/twisted-destiny/
FFF: DEBUG: 2022-08-29 10:26:33,803: fetcher.py(234):
========== MISS (GET) BasicCache
https://www.scribblehub.com/series/60207/twisted-destiny/
FFF: DEBUG: 2022-08-29 10:26:33,804: fetcher.py(469):
---------- REQ (GET) RequestsFetcher
https://www.scribblehub.com/series/60207/twisted-destiny/
FFF: DEBUG: 2022-08-29 10:26:33,891: fetcher.py(482): response code:403
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/fanficfare/fetcher.py", line 483, in request
    resp.raise_for_status() # raises RequestsHTTPError if error code.
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.scribblehub.com/series/60207/twisted-destiny/

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/fanficfare", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/fanficfare/cli.py", line 344, in main
    dispatch(options, urls, passed_defaultsini, passed_personalini, warn, fail)
  File "/usr/local/lib/python3.9/site-packages/fanficfare/cli.py", line 320, in dispatch
    do_download(url,
  File "/usr/local/lib/python3.9/site-packages/fanficfare/cli.py", line 435, in do_download
    adapter.getStoryMetadataOnly()
  File "/usr/local/lib/python3.9/site-packages/fanficfare/adapters/base_adapter.py", line 313, in getStoryMetadataOnly
    self.doExtractChapterUrlsAndMetadata(get_cover=get_cover)
  File "/usr/local/lib/python3.9/site-packages/fanficfare/adapters/base_adapter.py", line 417, in doExtractChapterUrlsAndMetadata
    return self.extractChapterUrlsAndMetadata()
  File "/usr/local/lib/python3.9/site-packages/fanficfare/adapters/adapter_scribblehubcom.py", line 120, in extractChapterUrlsAndMetadata
    data = self.get_request(url)
  File "/usr/local/lib/python3.9/site-packages/fanficfare/requestable.py", line 114, in get_request
    return self.get_request_redirected(url,
  File "/usr/local/lib/python3.9/site-packages/fanficfare/requestable.py", line 106, in get_request_redirected
    (data,rurl) = self.configuration.get_fetcher().get_request_redirected(
  File "/usr/local/lib/python3.9/site-packages/fanficfare/fetcher.py", line 398, in get_request_redirected
    fetchresp = self.do_request('GET',
  File "/usr/local/lib/python3.9/site-packages/fanficfare/fetcher.py", line 240, in fetcher_do_request
    fetchresp = chainfn(
  File "/usr/local/lib/python3.9/site-packages/fanficfare/fetcher.py", line 136, in fetcher_do_request
    fetchresp = chainfn(
  File "/usr/local/lib/python3.9/site-packages/fanficfare/fetcher.py", line 371, in do_request
    fetchresp = self.request(method,url,
  File "/usr/local/lib/python3.9/site-packages/fanficfare/fetcher.py", line 504, in request
    raise exceptions.HTTPErrorFFF(
fanficfare.exceptions.HTTPErrorFFF: HTTP Error in FFF '403 Client Error: Forbidden for url: https://www.scribblehub.com/series/60207/twisted-destiny/'(403)

[JimmXinu]

Dunno what's going on with that. It fails in FFF CLI, but works in the Calibre plugin from the same computer. I don't see any difference between the headers I can control.

[JimmXinu]

Seems to be working again. Going to call it something in the network or on the server. And I don't think there's anything I could do about it.

[mhussaincov93]

hi,

change your user agent in personal.ini.

the way i've done it is the folowing snip.

[www.scribblehub.com] slow_down_sleep_time:15 user_agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36 this user agent will work for scribble hub only.

please, please, don't hammer the site.

this is why i've set a large timeout between requests.

Majid

On 15/06/2023 21:03, Jack Evans wrote:

I'm getting this now too, I tried through a vpn too and no change.

— Reply to this email directly, view it on GitHub https://github.com/JimmXinu/FanFicFare/issues/876#issuecomment-1593656916, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASDFWN2CPPZXJMAGGAKOD3LXLNTADANCNFSM576S4NTQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>