nullxception / boorusphere

Simple, content-focused booru viewer for Android
BSD 3-Clause "New" or "Revised" License
395 stars 27 forks source link

it cant download from danbooru #150

Closed KiraiEEE closed 2 months ago

KiraiEEE commented 6 months ago

Type

Booru parser issues

Bug description

when danbooru selected and you like a pic to download it asks for quality and then doesn't show the downloading progress or even save it so i manually just click the link and get it from the browser

Steps to reproduce

  1. select danbooru
  2. select any picture
  3. click download icon
  4. select original or sample
  5. (no animation and you get failed notification)

Android version

android 12

App version

1.4.2

Screenshots or videos

Screenshot_2024-03-04-19-13-03-400_com miui home

https://github.com/nullxception/boorusphere/assets/54278089/6476946c-a6e1-4ab9-81c6-6f2152ef5a32

Solution

idk maybe use the link icon value to the downloader

Additional context

i hope this get fixed , i love the app and the ui and everything

Acknowledgements

KiraiEEE commented 6 months ago

btw the other completed ones are successfully downloaded using safebooru so the problem just with danbooru

makisukurisu commented 6 months ago

Can confirm. Also on Android 12. To be more precise - Google Pixel 3 with granted permission for files/media.

As stated in the description all the other (preinstalled) boorus (Gel-, Safe-, Konachan, Yandere) do work, as well as custom Gelbooru 0.2 beta boorus.

So, most likely the problem is related to how links are handled for danbooru specifically.

sadaakisz commented 5 months ago

The problem is indeed with Danbooru and how its cdn handles requests with headers.

While taking a look at #149 and scripting a personal solution for downloading all my favorites, I came across a 403 Forbidden when trying to download any image from Danbooru.

This forum thread indicates some funky business when trying to access content with a program that sends headers that make it look like a browser.

In the end I ended up sending requests without headers when downloading images and had no issues.

It may be worthwhile testing if removing the request headers when downloading from Danbooru solves the issue.

PS: If someone is interested in the fav downloader code, I'll leave it here: boorusphere_fav_dl

makisukurisu commented 5 months ago

Question for maintainers, perhaps this issue should be closed as a duplicate of #148? (Or vice versa)

makisukurisu commented 5 months ago

image

From my tinkering around, sending a request with no headers works just fine. The problem lies in the FlutterDownloader itself, it would seem. (And from debugging tools Flutter provides, its requests are also not visible)

Sadly, it's not possible to see the exact response, but it indeed has a 403 status code. I assume that this sender simply adds some data that doesn't “please” Danbooru → HTTP 403.

makisukurisu commented 5 months ago

Response Body

After some alterations to the source code of the DownloadWorker, it's now clear that the CDN presents some JS challenge to the downloads class, which it cannot solve.

Test with cURL And if you take a look at the source code, the reason is — it's sending a request with a "strange" User-Agent, which (most likely) got banned 😃

Possible ways of solving the problem:

Working request

nullxception commented 2 months ago

Duplicate of #148