search

ccloli / E-Hentai-Downloader

Download E-Hentai archive as zip file
GNU General Public License v3.0
1.82k stars 137 forks source link

Empty Response / TypeError: response is not an object #197

Open chikari802 opened 2 years ago

chikari802 commented 2 years ago

Ever since reinstalling this script on a new Linux machine, it has not been working anymore.

The issue seems to be an empty response (older versions say that but the new one shows a different error).

Please note that I am using Tor Browser 10.5.8 (based on Firefox 78.15.0esr) with customized settings.

Here's what I checked so far

Logs:

[EHD] UserAgent > Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0 E-Hentai Downloader.user.js:12318:9
[EHD] Script Handler > Violentmonkey E-Hentai Downloader.user.js:12319:9
[EHD] Script Handler Version > 2.13.0 E-Hentai Downloader.user.js:12320:9
[EHD] E-Hentai Downloader Version > 1.33.2 E-Hentai Downloader.user.js:12321:9
[EHD] Is Logged In > true E-Hentai Downloader.user.js:12323:9
[EHD] E-Hentai Downloader Setting > {"thread-count":1,"speed-detect":false,"number-images":false,"number-real-index":true,"number-auto-retry":true,"auto-download-cancel":true,"recheck-file-name":false,"ignore-torrent":false,"status-in-title":"never","hide-image-limits":true,"hide-estimated-cost":false,"file-descriptor":false,"force-resized":false,"never-new-url":false,"never-send-nl":false,"store-in-fs":false,"play-silent-music":false,"save-info":"file","save-info-list":["title","metas","uploader-comment","page-links"],"save-info-list[]":false,"replace-with-full-width":true,"force-pause":false,"save-as-cbz":false} E-Hentai Downloader.user.js:12282:11
[EHD] #1: Unknown Error (Please send feedback) E-Hentai Downloader.user.js:13411:13
[EHD] #1: RealIndex > 1  | ReadyState > 4  | Status > 200  | StatusText > OK
Request URL > https://exhentai.org/fullimg.php?gid=2019957&page=1&key=jow2mly9q7v 
Final URL > https://exhentai.org/fullimg.php?gid=2019957&page=1&key=jow2mly9q7v 
ResposeHeaders >Accept-Ranges: bytes

Age: 0

Content-Length: 0

Content-Type: text/html; charset=UTF-8

Date: Mon, 11 Oct 2021 19:54:20 GMT

Server: Apache

Strict-Transport-Security: max-age=31536000; preload;

Via: 1.1 varnish-v4

X-Varnish: 917565608

E-Hentai Downloader.user.js:13412:13
TypeError: response is not an object
    onload moz-extension://a1db2e9e-3535-4df5-ba3d-799ad8b75ef5/ E-Hentai Downloader.user.js#1:13090
    HttpRequested moz-extension://a1db2e9e-3535-4df5-ba3d-799ad8b75ef5/sandbox/injected-web.js:1
    HttpRequested moz-extension://a1db2e9e-3535-4df5-ba3d-799ad8b75ef5/sandbox/injected-web.js:1
    onHandle moz-extension://a1db2e9e-3535-4df5-ba3d-799ad8b75ef5/sandbox/injected-web.js:1
    c moz-extension://a1db2e9e-3535-4df5-ba3d-799ad8b75ef5/sandbox/injected-web.js:1
E-Hentai Downloader.user.js:13413:13
[EHD] Index > 1  | RealIndex > 1  | Name > 20210124_182650692_iOS.jpg  | RetryCount > 0  | DownloadedCount > 0  | FetchCount > 1  | FailedCount > 0 E-Hentai Downloader.user.js:12810:10
ccloli commented 2 years ago

Please try enabling Settings -> Advanced -> Pass cookies manually when downloading images to see if that helps. I don't see that option in your outputed settings, this should fixes sometimes the extension doesn't pass your cookies and received empty response. Though TypeError: response is not an object is a bit weird, too...

chikari802 commented 2 years ago

I have already tried enabling that option, forgot to mention it. It didn't make any difference.

ccloli commented 2 years ago

I noticed you're using an older version of Firefox, so maybe it's because the extension doesn't compatible with the old version. Have you ever tried other extensions like GreaseMonkey and Tampermonkey? You can install another extension and they should be able to run at the same time, just to make sure ViolentMonkey may need to be disabled when installing the script, and don't forget to enable it back.

chikari802 commented 2 years ago

It could be related to the version, but since Tor Browser uses Firefox ESR, it's still on version 78. The new version of Firefox ESR is 91, maybe they will update to it when 78 is no longer supported.

With Tampermonkey, it gets stuck at "Downloading 0 KB/s" (no error), and a warning appears in the console:

[EHD] UserAgent > Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0 E-Hentai Downloader.user.js:12320:9
[EHD] Script Handler > Tampermonkey E-Hentai Downloader.user.js:12321:9
[EHD] Script Handler Version > 4.13.6136 E-Hentai Downloader.user.js:12322:9
[EHD] E-Hentai Downloader Version > 1.33.2 E-Hentai Downloader.user.js:12323:9
[EHD] Current URL > https://exhentai.org/g/2019957/387aa81f22/?p=2 E-Hentai Downloader.user.js:12324:9
[EHD] Is Logged In > true E-Hentai Downloader.user.js:12325:9
[EHD] E-Hentai Downloader Setting > {"thread-count":1,"speed-detect":false,"number-images":false,"number-real-index":true,"number-auto-retry":true,"auto-download-cancel":true,"recheck-file-name":false,"ignore-torrent":false,"status-in-title":"never","hide-image-limits":true,"hide-estimated-cost":true,"file-descriptor":false,"force-resized":false,"never-new-url":false,"never-send-nl":false,"never-warn-large-gallery":false,"store-in-fs":false,"play-silent-music":false,"save-info":"file","save-info-list":["title","metas","uploader-comment","page-links"],"save-info-list[]":false,"replace-with-full-width":false,"force-pause":false,"save-as-cbz":false,"pass-cookies":true,"force-as-login":false} E-Hentai Downloader.user.js:12284:11
TypeError: NetworkError when attempting to fetch resource.
    o moz-extension://fccce35c-e313-45b1-a2a5-006abe3aaeae/content.js:103
    n moz-extension://fccce35c-e313-45b1-a2a5-006abe3aaeae/content.js:103
    xhr moz-extension://fccce35c-e313-45b1-a2a5-006abe3aaeae/content.js:104
content.js:103:485

With GreaseMonkey, it gets stuck at "Pending", with no errors or warnings in the console.

ccloli commented 2 years ago

If it comes to NetworkError, then there is nothing I can do with that. The error seems from the browser or extension itself, since I don't see the log from the script.

Maybe it's because E-Hentai is mostly a P2P site, the image is hosted on the devices running Hentai@Home, so for Tor it maybe hard to connect these devices after passing multiple nodes. Or maybe the extension doesn't compatible with Tor browser, they can't send the request so it ends with NetworkError.

Maybe you can try other tools like the CLI program https://GitHub.com/mikf/gallery-dl , if you do care about the privacy concerns, you can run a global VPN client.

TankedThomas commented 2 years ago

I'm having the same problem with Tampermonkey after Waterfox was updated to the latest version today (G4.0.0.1 (64-bit)). It was working fine last night before the update to Waterfox G4.

[EHD] E-Hentai Downloader is running. E-Hentai Downloader.user.js:39:9
[EHD] Bugs Report > https://github.com/ccloli/E-Hentai-Downloader/issues | https://greasyfork.org/scripts/10379-e-hentai-downloader/feedback E-Hentai Downloader.user.js:40:9
[EHD] To report a bug, it's recommended to provide the logs started with "[EHD]", thanks. =w= E-Hentai Downloader.user.js:41:9
[EHD] UserAgent > Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Firefox/91.0 Waterfox/91.3.0 E-Hentai Downloader.user.js:12320:9
[EHD] Script Handler > Tampermonkey E-Hentai Downloader.user.js:12321:9
[EHD] Script Handler Version > 4.13.6136 E-Hentai Downloader.user.js:12322:9
[EHD] E-Hentai Downloader Version > 1.33.2 E-Hentai Downloader.user.js:12323:9
[EHD] Current URL > https://exhentai.org/g/509591/3bd5b912e6/ E-Hentai Downloader.user.js:12324:9
[EHD] Is Logged In > true E-Hentai Downloader.user.js:12325:9
[EHD] E-Hentai Downloader Setting > {"speed-detect":false,"number-images":false,"number-real-index":true,"number-auto-retry":true,"auto-download-cancel":true,"dir-name":"{title}","file-name":"{title}","recheck-file-name":false,"ignore-torrent":true,"status-in-title":"blur","hide-image-limits":false,"hide-estimated-cost":false,"file-descriptor":false,"force-resized":false,"never-new-url":false,"never-send-nl":false,"never-warn-large-gallery":true,"store-in-fs":false,"play-silent-music":true,"save-info":"none","save-info-list":["title","metas","uploader-comment","page-links"],"save-info-list[]":false,"replace-with-full-width":false,"force-pause":false,"save-as-cbz":false,"pass-cookies":true,"force-as-login":false} E-Hentai Downloader.user.js:12284:11
[EHD] Request Resolution Setting E-Hentai Downloader.user.js:14555:10
[EHD] Request Image Limits From e-hentai.org E-Hentai Downloader.user.js:14437:10
[EHD] Resolution Setting > {"withoutHentaiAtHome":0,"resolution":0,"timestamp":1634809475284} E-Hentai Downloader.user.js:14566:11
TypeError: NetworkError when attempting to fetch resource.
    o moz-extension://edc72006-1ab7-4a95-ba54-7aff0f3e612d/content.js:103
    n moz-extension://edc72006-1ab7-4a95-ba54-7aff0f3e612d/content.js:103
    xhr moz-extension://edc72006-1ab7-4a95-ba54-7aff0f3e612d/content.js:104

I tried reinstalling the script and tried passing cookies manually (you can see it's enabled in my log) but it doesn't fix the problem. When I tried forcing the download, I got a blank archive.

ccloli commented 2 years ago

I'm having the same problem with Tampermonkey after Waterfox was updated to the latest version today (G4.0.0.1 (64-bit)). It was working fine last night before the update to Waterfox G4.

Okay, this can be reproduced on a fresh new installed Waterfox, though I'm still confused what's up, since the error comes from the extension itself.

[EHD] #3: Network Error E-Hentai Downloader.user.js:13434:12
[EHD] #3: RealIndex > 3  | ReadyState > 4  | Status > 0  | StatusText > 
Request URL > ***
Final URL > ***
ResposeHeaders > E-Hentai Downloader.user.js:13435:12
[EHD] Index > 3  | RealIndex > 3  | Name > 02.jpg  | RetryCount > 0  | DownloadedCount > 0  | FetchCount > 5  | FailedCount > 0 E-Hentai Downloader.user.js:12812:10
TypeError: NetworkError when attempting to fetch resource.
    o moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:103
    n moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:103
    xhr moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:104
content.js:103:485
TypeError: NetworkError when attempting to fetch resource.
    o moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:103
    n moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:103
    xhr moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:104
content.js:103:485
TypeError: NetworkError when attempting to fetch resource.
    o moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:103
    n moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:103
    xhr moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:104
content.js:103:485
TypeError: NetworkError when attempting to fetch resource.
    o moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:103
    n moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:103
    xhr moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:104
content.js:103:485
TypeError: NetworkError when attempting to fetch resource.
    o moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:103
    n moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:103
    xhr moz-extension://4d484a19-8947-472d-b940-85e506e3e999/content.js:104
content.js:103:485
ccloli commented 2 years ago

Here is the log from the browser console (by pressing Ctrl + Shift + J) when using Tampermonkey, and seems that Waterfox has some security limit that the extension doesn't bypass. Though it seems that the extension is loading the data from the same origin, so the browser shouldn't block the request.

Security Error: Content at moz-extension://4d484a19-8947-472d-b940-85e506e3e999/ may not load data from blob:moz-extension://4d484a19-8947-472d-b940-85e506e3e999/21783c76-b2af-489d-85eb-778f8c52c236.
Security Error: Content at moz-extension://4d484a19-8947-472d-b940-85e506e3e999/ may not load data from blob:moz-extension://4d484a19-8947-472d-b940-85e506e3e999/1bc696f6-7769-4eec-85b5-d40fda68c6cc.
Security Error: Content at moz-extension://4d484a19-8947-472d-b940-85e506e3e999/ may not load data from blob:moz-extension://4d484a19-8947-472d-b940-85e506e3e999/94c24ae2-f8a1-4896-835a-b597cdd63765.
Security Error: Content at moz-extension://4d484a19-8947-472d-b940-85e506e3e999/ may not load data from blob:moz-extension://4d484a19-8947-472d-b940-85e506e3e999/38b0c670-69a1-4832-8d2e-80454c67447f.
Security Error: Content at moz-extension://4d484a19-8947-472d-b940-85e506e3e999/ may not load data from blob:moz-extension://4d484a19-8947-472d-b940-85e506e3e999/b2129c43-7576-4a6f-a5bd-73b026a7a122.

For GreaseMonkey on Waterfox, I can see there is another error that cause the problem. It seems that the extension wants to read cookies but the browser doesn't return.

cookies is undefined on-user-script-xhr.js:129
    open moz-extension://bf8f1a29-cd63-4108-8f02-f38343a94dec/src/bg/on-user-script-xhr.js:129
    apply self-hosted:2736
    applySafeWithoutClone resource://gre/modules/ExtensionCommon.jsm:626
    wrapPromise resource://gre/modules/ExtensionCommon.jsm:855
    withLastError resource://gre/modules/ExtensionCommon.jsm:774
    wrapPromise resource://gre/modules/ExtensionCommon.jsm:843
Unchecked lastError value: Error: First-Party Isolation is enabled, but the required 'firstPartyDomain' attribute was not set. on-user-script-xhr.js:128
    open moz-extension://bf8f1a29-cd63-4108-8f02-f38343a94dec/src/bg/on-user-script-xhr.js:128

For ViolentMonkey, it works well on Waterfox, I can get the downloaded files without any problem. @TankedThomas I think you should give it a try.

@chikari802 Your issue is not the same as the previous comments and the section above, so you can skip that.

I also tried the Tor Browser, and sorry to say that I'm still cannot reproduce the bug. The script can download the gallery on Tampermonkey without any issue, so I'm still have no clue what's going on.

If you still want to use the script, my final idea is to add E-Hentai to Cookies policy's whitelist:

  1. Go to about:preferences#privacy
  2. Find Cookies and Site Data section, and click Manage Permissions... button
  3. On the open dialog, find the input box of Address of website, type e-hentai.org and exhentai.org, then click the Allow button
  4. Click Save Changes button to close the dialog

You can also try pressing Ctrl + Shift + J and start downloading, and see if the browser's console shows any useful log.