RedSquirrel87 / manga-downloader

A very simple application written in JAVA to download any chapters of any manga from the supported websites
37 stars 1 forks source link

bato.to #530

Closed RedSquirrel87 closed 3 years ago

RedSquirrel87 commented 3 years ago

Original report by Anonymous.


bato.to not working

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


It seems they enabled a cloudflare/recaptcha protection, we need to wait they disable it (usually this happens in some days)

RedSquirrel87 commented 3 years ago

Original comment by masa-chan (Bitbucket: strawberryturtle501, ).


Do you know if they’re going to disable this anytime soon? Also, I’m assuming there’ll be a new version of the downloader once Mangadex v.5 is released again?

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


I don’t think they will disable it this time, it’s about a month they enabled it and it’s still there. Anyway next major update of the program will include support for an external web browser extension usefull to bypass the anti-bot protections like Cloudflare and similar, so batoto will return to work again

RedSquirrel87 commented 3 years ago

Original comment by masa-chan (Bitbucket: strawberryturtle501, ).


okayy, I await for the next update! Thanks for the hard work.

Btw, is there any way I can be notified of updates? Like discord or twitter that you usually post on??

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


Upgraded to the new version and just getting an ERROR when it tries to load chapter list when using Bato.to links.

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


Are you using the new method to bypass the anti-bot protection? You need to use the web browser extension: https://redsquirrel87.com/mda
P.s.: please note also that you need to force the anti-bot protection from the specific button in the extension (Batoto needs it)

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


Just to be sure I tested it again right now and it works for me here:

So please be sure to follow the procedure correctly (remember to force the anti-bot protection with the specific button of the extension since Batoto usually don’t show the protection from known web browsers)

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


I am using the .jar on my Mac. Tried installing the browser extension on Chrome and copied the data, but where am I pasting it? If it is in the part where we usually enter manga URL it just infinitely loads.

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


Wait, are you using the v50 of the app? It should appear a specific window with all the instructions 🤔

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


This one:

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


Yep. Auto-upgraded to v50 and download the missing components.

The Chrome extension gives me the pop-out window with the data needed as expected, but nothing appears on the Manga Download app, if it is supposed to automatically load.

Am I supposed to add a Bato.to series link to Manga Downloader before doing that step? What brings up that Bypass anti-bot protections window in the app.

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


After you insert the link in the main window here:

by pressing OK you should get this window:

Doesn't it show up to you?

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


If it does not appear to you please upload/paste here the jvm_log.txt file content, so that I can see if there is any error on Mac that prevents the procedure to work (I don’t have any Mac here so I can’t test by myself)

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


[Fri Apr 23 23:24:46 AEST 2021] Program started.
[Fri Apr 23 23:24:46 AEST 2021] MangaDownloader v50
[Fri Apr 23 23:24:46 AEST 2021] OS: Mac OS X (x86_64) | JVM: 1.8.0_121
[Fri Apr 23 23:24:57 AEST 2021] Loading manga from https://bato.to/series/84970/
[Fri Apr 23 23:24:58 AEST 2021] HTML response code: 200
[Fri Apr 23 23:24:58 AEST 2021] Framework: BATOTO
[Fri Apr 23 23:24:59 AEST 2021] 0 chapters found.
[Fri Apr 23 23:25:20 AEST 2021] Loading manga from https://bato.to/series/84970/i-ll-just-live-on-as-a-villainess-bored-corona-kids-version
[Fri Apr 23 23:25:21 AEST 2021] HTML response code: 200
[Fri Apr 23 23:25:21 AEST 2021] Framework: BATOTO
[Fri Apr 23 23:25:22 AEST 2021] 0 chapters found.

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


By the way, this screen below also loads weird off the screen and can’t be resized. It is like three screen widths long.

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


Well, it’s the “HTML response code: 200” the issue, when you get the Cloudflare protection the server should return 40x or 50x, so the program does not show the anti-bot window because you does not seem to be blocked by the website 🤔

Please could you test to load this link https://toonily.com/webtoon/god-of-blackfield/ and paste here the jvm_log.txt file content? It’s a website with another Cloudflare protection (more aggressive than batoto) and I would like to see if you get a correct http response for it or maybe if it’s all different on Mac

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


Seems to work fine for Toonily:

[Fri Apr 23 23:45:15 AEST 2021] Program started.
[Fri Apr 23 23:45:15 AEST 2021] MangaDownloader v50
[Fri Apr 23 23:45:15 AEST 2021] OS: Mac OS X (x86_64) | JVM: 1.8.0_121
[Fri Apr 23 23:45:25 AEST 2021] Loading manga from https://toonily.com/webtoon/god-of-blackfield/
[Fri Apr 23 23:45:25 AEST 2021] HTML response code: 503
[Fri Apr 23 23:46:05 AEST 2021] HTML response code: 200
[Fri Apr 23 23:46:05 AEST 2021] Framework: TOONILY
[Fri Apr 23 23:46:06 AEST 2021] 76 chapters found.

That window loaded fine for me to paste the data from the Chrome extension into and the chapter list generated correctly.

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


Thank you for your test, so the issue is why batoto returns the code 200 🤔

If you want to help debugging this issue, I’ll give you a special version that will print in the jvm_log.txt the HTML response of batoto so that we can discover what it returns, just a couple of minutes to compile it and I’ll upload it here

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


Yep, sounds good.

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


Thank you very much :slight_smile:

This is the debug version: https://bitbucket.org/Red_Squirrel/manga-downloader/downloads/debug.jar
Just put it in the same folder where is the program (otherwise it’ll download again all the libs xD) and run it (java -jar debug.jar). Try to load any batoto link and it should write the batoto HTML response inside the txt log file

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


Here it is, although I must say that Bato.to functioned correctly in this build you provided and the window popped up and I could generate the chapter list:

[Sat Apr 24 00:05:19 AEST 2021] Program started.
[Sat Apr 24 00:05:19 AEST 2021] MangaDownloader v50
[Sat Apr 24 00:05:19 AEST 2021] OS: Mac OS X (x86_64) | JVM: 1.8.0_241
[Sat Apr 24 00:05:34 AEST 2021] Loading manga from https://bato.to/series/84970/i-ll-just-live-on-as-a-villainess-bored-corona-kids-version
[Sat Apr 24 00:05:34 AEST 2021] HTML response code: 403
[Sat Apr 24 00:05:59 AEST 2021] HTML response code: 200
[Sat Apr 24 00:05:59 AEST 2021] Framework: BATOTO
[Sat Apr 24 00:05:59 AEST 2021] <!DOCTYPE html>

``` I'll Just Live on as a Villainess [Bored Corona Kids Version] Manga ``` ```
```
그냥 악역으로 살겠습니다
>
Rank: 68th, it has 40330 monthly / 42935 total views.
Authors: Kim da-ham / Bebe
Genres: Korean , Manhwa , Webtoon , Shoujo , Demons , Fantasy , Isekai , Magic , Reincarnation , Romance , Supernatural
Read direction: Left to Right
Release status: Ongoing
Year of Release: 2021
MPCRS( target="_blank">?): A - No age restrictions
Summary:
I reincarnated as the villainess in a book!
```

The one who dared to commit attempted murder on the heroine is the owner of this body?

Let’s just live in a quiet place where we have fun and eat! That’s what I thought for a while.  
It was so, so, so cold here in the north, where I was kicked out as a punishment.  
Before I froze to death, I called the Great Demon of Fire and set fire to the fireplace but...

Why isn’t he going back? If you’ve done the job, shouldn’t you go back?!  
I was flustered to find out that I had signed a life contract with a demon just to start a fire but to think that I’d be responsible for relieving his desires!

The bickering romance between a big puppy demon and a small villainess lady!
```
Resources:
>

    

Chapters (26)

${history._chapter._extra.volcha} ${history._visitDate}
```
```

Comments

Discussions

```
```
```

[Sat Apr 24 00:05:59 AEST 2021] 26 chapters found.

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


Well this is a mystery xD Please try to delete the main.jar and restart the program (it will download the main.jar again) and see if now it works also for the “normal” program

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


Did as you instructed. Still fails when it re-dowloads the main.jar.

I might just download a fresh install of the program and see if it works. Potentially mine may have got corrupted.

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


Tried fresh install, but .jar force quits as soon as it opens.

Went back to old one and now even debug.jar you provided fails to load window.

Here is output, with the html:

[Sat Apr 24 00:44:19 AEST 2021] Program started.
[Sat Apr 24 00:44:19 AEST 2021] MangaDownloader v50
[Sat Apr 24 00:44:19 AEST 2021] OS: Mac OS X (x86_64) | JVM: 1.8.0_291
[Sat Apr 24 00:44:22 AEST 2021] Loading manga from https://bato.to/series/82947/rumor-has-it
[Sat Apr 24 00:44:23 AEST 2021] HTML response code: 200
[Sat Apr 24 00:44:23 AEST 2021] Framework: BATOTO
[Sat Apr 24 00:44:23 AEST 2021] <!DOCTYPE html>

Please Wait... | Cloudflare
<script type="text/javascript">
//<![CDATA[
(function(){
  window._cf_chl_opt={
    cvId: "2",
    cType: "managed",
    cNounce: "77162",
    cRay: "6447de9f397916b9",
    cHash: "ae585734fa14330",
    cFPWv: "g",
    cTTimeMs: "4000",
    cLt: "n",
    cRq: {
      ru: "aHR0cHM6Ly9iYXRvLnRvL3Nlcmllcy84Mjk0Ny9ydW1vci1oYXMtaXQ=",
      ra: "TW96aWxsYS81LjAgKFdpbmRvd3MgTlQgMTAuMDsgV2luNjQ7IHg2NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzg3LjAuNDI4MC4xNDEgU2FmYXJpLzUzNy4zNg==",
      rm: "R0VU",
      d: "Pd1VmnwrsOem9qUizemqbDnl6eOPZz0d/qIjgxo3mAV70KCKRmT1gfbGvkkn34z//NQ82/2GRCOwGxYZZPRJg5gDIcZDVcNcv+IZXc3uIXpghwy3NyR3kTGFCdh1D2Zm+Xl+jn6HB357NkCVm/rj0i9f/c+NH2rwKwM62+uM12E9qZeWuLMySiwcYacdEWO72fr10paCIMPBwIi8aKZw5IixLdeqAoK5T74yLuxkMdUApuGCFxZR4nI/MLopm8Ur/0t165tdY2i35uMXKkDOaJy5hD/6NtvO0UVgAHZ9wjTTCn65mOHTGN7FBCSEwEZAGDWkgroKm29CSEyUNQvimD4smuvgQ0rFsPi1Ijg1bfkpUwOOyaYAA8AwQBEq9fEa7/UFEopHeo7dLb67sbxMUKYGvIzoAa1/HVF2CtVGErPosSUr8Wo1gAlnSQy3nbPzWyUgpVQefcWVwo4A2GbaQq/gCJrkxTW89FbTGGJMAHVQj8dSNQmd2cM6yblSn0e8XHnZQXkUEep6mDouDg3/R91kKKIPYtOOQNlEU6YQD7DWTRW13suI6bkJ4fQsnLjbNu/mCzF7YYj2vAWNRWsVGJ8XAf33z8L+FzB1ILSTYzrpqah65sWrXcROreuwVeeuz9kbVttQcqk2mZ29h1VGbeBI+rM/G8rA/xdWN52aV9X0X1VPcsFuacYdVrifGjmNtQi/bZK/iQM7xQb2gVAP0eMV6FGWA4A+fPOGBK6Qe8spkB/+ouM+dG2JIH3lrCWvxp/p16pTe4g4KO//d0dmv7y1DxXptEqr/wklJ3Aiy6oDOmVxFoRNL5Tz/qEjkV4hRrI7lDZ4MPfrH/yR4qnHJrUmEC/Eu4uTuJFnwP9AoBifMCgF2+oPsKFwWO4/iRnlmtTCumrypPUdAY5V969bENTVFbXalKgvMcfGYBtAPh0=",
      t: "MTYxOTE4OTA2My41NTcwMDA=",
      m: "Oxfw7paaA5TDeHll2FVS0jhKyN54UGsotHqjhWPs7qs=",
      i1: "k6jFCRlzniCenIb5CoccCw==",
      i2: "H9ixp2mzowqP/gnS09vaTA==",
      zh: "JjmHkKw5rN5etxEEa1nun2OjlYYjJlOyuJ0vmKO5to0=",
      uh: "sfgiHR/bGP0DFXSp4KJXpJnmQXuNHfWxwUhpX9SCMz4=",
      hh: "ND2Zn2Yu7n5kQNkhsZ6GzYsKr8lcpMXa+V2p9RuGiDI=",
    }
  };
}());
//]]>
</script>

```

Please wait...

We are checking your browser... bato.to

Please stand by, while we are checking your browser...

```
```

Why do I have to complete a CAPTCHA?

Completing the CAPTCHA proves you are a human and gives you temporary access to the web property.

What can I do to prevent this in the future?

If you are on a personal connection, like at home, you can run an anti-virus scan on your device to make sure it is not infected with malware.

If you are at an office or shared network, you can ask the network administrator to run a scan across the network looking for misconfigured or infected devices.

Another way to prevent getting this page in the future is to use Privacy Pass. You may need to download version 2.0 now from the Chrome> Web Store.

```
```

[Sat Apr 24 00:44:23 AEST 2021] 0 chapters found.

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


The .jar force quitting was resolved by moving Manga Downloader to my Applications folder. It appears the permissions for .jar or JRE are different from the Downloads folder.

Anyway, with a clean install with the new version 50.1, but it is still failing for bato.to and also seems to be replicated for Webtoons and Mangapark (although instead of along the top right where it usually says Error, it brings a little message “We are checking your browser… mangapark.net” for indefinite time.

Problem may be occurring across more sites where it is failing to load the window, but no time to check at the moment.

PS: The GUI problem is fixed on first load, thanks for that quick update.

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


Webtoons is currently broken, it needs a fix. Mangapark uses the same anti-bot protection of Bato.to and so the error. The problem is caused by that http response code (200) which basically tells the program that there are no problems accessing the site (when instead there is the Cloudflare protection in the way). I really have no idea why the server returns that wrong http response code 🤔
The only solution I can think is to insert an option in the input window to force the bypass method to be activated without relying on the response code

RedSquirrel87 commented 3 years ago

Original comment by Red Squirrel (Bitbucket: Red_Squirrel, GitHub: RedSquirrel87).


I just released the v50.2 which adds the option to force the bypass method and should fix webtoons support too

RedSquirrel87 commented 3 years ago

Original comment by Jack Dowl (Bitbucket: [Jack Dowl](https://bitbucket.org/Jack Dowl), ).


Just wanting to thank you for fixing the problem in the new release v50.2 with the Force bypass method. Using that method in conjunction with forcing anti-bot protection allows it to work.