Can not donwload media by using contextual menu at Pixiv and other sites anymore

LuckyTurtleDev commented 3 years ago

Steps to reproduce

open https://www.pixiv.net/en/artworks/87369838#big_0 on Firefox for Android (you may will be redirected to an login page. Try to open it again or to login before opening the link. A account can be created for free)
long press on the picture
tap on save image

Expected behavior

Picture should be downloaded

Actual behavior

Message appears which says: "Download fehlgeschalgen" (Download failed)

Device information
Sony Xperia XZ1 COMPACT (G8441)
Stock-ROM Android 9
Firefox 85.1.1, Firefox Beta 85.0.0-beta.9 and Fennec F-Droid 85.1.0

Additional information

For a few days I was still able to download the image from Pixiv. Furthermore the old Fennec F-Droid 84.1.4 is still able to download the images. May something with the version 85 is wrong? I also try to download image from other website, but I was not able to reproduce this issue with other sites.

Before this issue I also notice some other strange behavior while downloading image:

Firefox does not react after download image for roughly 1 min. This often happen after starting the app and does not happen again after some time/downloads.
The download failed every time, if i close the tab before the download has finished.
If I tab long on a image some times no windows is open, other times my cell phone vibrate too. But some times later it does works again.

Depends on

[ ] https://bugzilla.mozilla.org/show_bug.cgi?id=1690037

┆Issue is synchronized with this Jira Task

LittleAngelwings commented 3 years ago

Download fails also with latest nightly from Play Store.

Mugurell commented 3 years ago

I see this failing on all current releases. Trying a GET for https://i.pximg.net/img-original/img/2021/01/29/00/18/38/87369838_p0.png (the image from the address in the report) returns 403 - Forbidden. After a quick search on the Internet it seems that this website may need some special treatment. Seeing that the download works on Chrome and on Firefox desktop but fails with 403 on GeckoViewFetchClient maybe there is something that we can do. Maybe @Amejia481 knows more.

LuckyTurtleDev commented 3 years ago

I have try to open the https://i.pximg.net/img-original/img/2021/01/29/00/18/38/87369838_p0.png from Firefox Desktop and it does also return a 403. But if I open https://www.pixiv.net/en/artworks/87369838#big_0 and click on view image, Firefox Desktop open https://i.pximg.net/img-original/img/2021/01/29/00/18/38/87369838_p0.png and it does work.

Amejia481 commented 3 years ago

I see this failing on all current releases. Trying a GET for https://i.pximg.net/img-original/img/2021/01/29/00/18/38/87369838_p0.png (the image from the address in the report) returns 403 - Forbidden. After a quick search on the Internet it seems that this website may need some special treatment. Seeing that the download works on Chrome and on Firefox desktop but fails with 403 on GeckoViewFetchClient maybe there is something that we can do. Maybe @Amejia481 knows more.

We have some API limitations when we download via long press, these downloads don't use our normal download stack, instead they use GeckoViewFetchClient(GeckoWebExecutor) as a result, some metadata about the download get lost in the process causing this and similar issues. I will file an issue for the GeckoView team to explore if is possible for them to expose the same API via contextual menus.

Amejia481 commented 3 years ago

GeckoView ticket https://bugzilla.mozilla.org/show_bug.cgi?id=1690037

iDawer commented 3 years ago

Seems like Pixiv is blocking requests without a referer. This shell command works fine:

curl -H "Referer: https://www.pixiv.net/" https://i.pximg.net/img-original/img/2021/01/29/00/18/38/87369838_p0.png

lobontiumira commented 3 years ago

Reproducible on RC 88.1.4 with Samsung Galaxy Note 8 (Android 9).

lobontiumira commented 3 years ago

Reproducible on Beta 90.0.0-beta.2 with HTC 10 (Android 8).

varishtsg commented 3 years ago

Facing the same issue on Firefox Android on Poco F1 (Android 11)

89.1.1 (Build #2015812947) AC: 75.0.22, 5204f4025 GV: 89.0-20210527174632 AS: 74.0.1

eqyliq commented 3 years ago

Same here on 89.1.1 with a P40 Pro (android 10, emui 11.0.0.193)

shuntress commented 3 years ago

I found this existing issue while creating a new issue for a bug that I believe has the same underlying problem.

Attempting to download (through the long press context menu) a resource that requires HTTP Basic Authentication fails. Firefox makes a new request for the resource without including the auth header, the server sends back a 401 or 403, Firefox then considers the download failed.

It would be better if the browser re-sends the request with the appropriate auth info (prompting the user if necessary.)

In my case, this is with Firefox Daylight 91.4.0 (Build #2015831131)

43615 commented 3 years ago

Still happens on 93.2.0. Seeing no progress on this is baffling, this issue is seriously impacting usability for me.

CaFI0 commented 3 years ago

This problem still persists on my Firefox Nightly 95.0a1 (Build # 2015841481)

mrxyzl commented 2 years ago

This issue still persist on latest nightly, 96a1. This is a serious bug as it hinder browser operation and yet take too long to be fixed

CaFI0 commented 2 years ago

This problem still persists on the latest nightly 98.0a1, it is almost a year since this issue was opened.

plwt commented 2 years ago

User feedback: https://support.mozilla.org/en-US/questions/1364880

designzwang commented 2 years ago

I found this existing issue while creating a new issue for a bug that I believe has the same underlying problem.

Attempting to download (through the long press context menu) a resource that requires HTTP Basic Authentication fails. Firefox makes a new request for the resource without including the auth header, the server sends back a 401 or 403, Firefox then considers the download failed.

It would be better if the browser re-sends the request with the appropriate auth info (prompting the user if necessary.)

In my case, this is with Firefox Daylight 91.4.0 (Build #2015831131)

@shuntress As of 2022-01: experienced the same behaviour with both latest Firefox (on Samsung S10/Android12) and Fennec (F-Droid/Lineage 18.1/Nexus7). Checked server logs - if request is initiated via context menu - "download link" then no AUTH headers are sent and download fails without even asking for user/password.

ctrlcctrlv commented 2 years ago

I am not sure that the explanation given here makes any sense. The data for the image is already in the browser's memory. Why is another request even being sent? For some images and some websites that might cause you to get a totally different image, what if the end point is programmed to return a random image?

So why not just take the image that's in memory and save it... If that can't be done that's the real API problem. Imagine that a user has an image open in their browser and then the server goes down, and that image turns out to be important... Again they can't save it.

So this isn't just a Pixiv issue, it's an issue in the entire way you have conceived of downloading by right click in the mobile browser.

LuckyTurtleDev commented 2 years ago

Maybe their is no option to get the image from GeckoView. So they simple redownload it. But I agree a option to save the image without redownloading should be the better solution.

areyance commented 2 years ago

problem still persists on 99.2.0 ver

ctrlcctrlv commented 2 years ago

Maybe their is no option to get the image from GeckoView. So they simple redownload it.

OK, if that's the case, that's the real problem, and needs fixed there. I'm trying to move this bug forwards, the current "solution" isn't one and more sites than Pixiv are broken (you should update the title, @lukas1818).

gaurav5125 commented 2 years ago

Is any developer working on this?? Downloading from Pixiv still doesn't work on any of the Firefox.

LuckyTurtleDev commented 2 years ago

Is any developer working on this?? Downloading from Pixiv still doesn't work on any of the Firefox.

Current no. The Gecko Ticket is unassigned again and the priority was reduce.

ctrlcctrlv commented 2 years ago

I challenged the decrease in Gecko bug priority in Ticket № 1690037 §10. Who knows if they will listen, though.

(In reply to Chris Peterson [:cpeterson] from comment #9)

This is a huge problem for Fenix, not just nice to save. Fenix downloading is totally broken. I do not understand why you bumped down the priority, [:cpeterson].

This does not just affect Pixiv. As I wrote on GitHub:

The data for the image is already in the browser's memory. Why is another request even being sent? For some images and some websites that might cause you to get a totally different image, what if the end point is programmed to return a random image?

So why not just take the image that's in memory and save it... If that can't be done that's the real API problem. Imagine that a user has an image open in their browser and then the server goes down, and that image turns out to be important... Again they can't save it.

So this isn't just a Pixiv issue, it's an issue in the entire way you have conceived of downloading by right click in the mobile browser.

There are so many bugs that can be caused by doing it the way it's done right now in Fenix…even security related ones if webmasters catch on and tamper with downloaded files only when downloaded by this known bad browser…please fix it.

This issue is serious, and even if users (including myself, you all know, "does the furry go on Pixiv"—does the raccoon shit in the woods?) are only seeing it because they go on the weird Japanese cartoon porn site (Pixiv) does not mean that the bug only affects Pixiv and can be ignored, it means that we found a serious bug because of a webserver that plays games with the browser. The furries are helping, I mean. Some call Firefox "furryfox", well, "let God be true and every man a liar" :joy_cat:

MikuAuahDark commented 2 years ago

Sorry for hijacking, but I don't think S3 severity is suitable for this. Are there actually workaround for this issue beside switching browser? I don't think so. Quoting the Mozilla wiki about severity:

S3: (Normal) Blocks non-critical functionality and a work around exists

lindongbin commented 2 years ago

I think Firefox developers don't use Firefox, so don't know what users really need features and bugs that need to be fixed

designzwang commented 2 years ago

Still no solution, this was last tried with Firefox on an Samsung S10 with 104.1.0 (Build #2015898515), 99759dd72+ AC: 104.0.8, 3dc0ff00e7 GV: 104.0-20220816115024 AS: 93.7.2

In my case it is not images, but MP3s - I am accessing a list of audio files behind basic auth. While being able to browse, and also to play the files in the builtin player, trying to save or download fails. This is the reason, shown in a simplified log entry: The mp3 request has no Auth Header any more. 11.22.33.44 - username [05/Sep/2022:00:20:57 +0200] "GET /listing HTTP/1.1" 200 33633 "Mozilla/5.0 (Android 12; Mobile; rv:104.0) Gecko/104.0 Firefox/104.0" 11.22.33.44 - - [05/Sep/2022:00:21:07 +0200] "GET /file.mp3 HTTP/1.1" 401 723 "-" "Mozilla/5.0 (Android 12; Mobile; rv:104.0) Gecko/104.0 Firefox/104.0"

@MikuAuahDark - I switched to Chrome just because of this bug, no other workaround that I know of @lindongbin - This type of comment is highly inappropriate. If you were a developer, would you feel motivated to even spend a single second of your time for people with this attitude ?

Igetin commented 2 years ago

I'm personally using a Bash script workaround which relies on Termux. Here's a short guide if anyone wants to do the same for Pixiv images.

Set up Termux (if you haven't already)

Download and install the Termux app
Set up the package manager by running termux-setup-package-manager
Set up the storage permissions by running termux-setup-storage
Install Wget by running pkg install wget
- technically not required for downloading stuff via HTTP since cURL should be pre-installed, I just personally prefer wget for this purpose
Allow Termux to draw over other apps
- not strictly necessary, but can offer a nicer experience when running scripts via the Android share menu
- the exact way to do this varies on your Android version and your OEM-specific overlay

Install the script

Create a file in the path ~/bin/termux-url-opener
- create the directory if it doesn't exist: mkdir ~/bin
- edit the file, e.g. nano ~/bin/termux-url-opener
Add the following contents to the file:
Show script

#!/data/data/com.termux/files/usr/bin/bash
if [[ $1 = https://i.pximg.net* ]]
then
    wget --referer https://pixiv.net --directory-prefix ~/storage/shared/Download $1
    return_code=$?
    if [ $return_code -eq 0 ] 
    then 
        echo 'Image downloaded!'
    else 
        echo 'Failed to download the image.' >&2 
    fi
else
    return_code=0
    echo 'Not a Pixiv image URL.'
fi
echo 'Returning to the previous app.'
sleep 1.5
exit $return_code

Make the file executable: chmod +x ~/bin/termux-url-opener

Now, when you long-press a Pixiv image in Firefox, you can select the Share link option and then select Termux from the menu. This will open Termux, run the script to download the image to your downloads folder, and return to Firefox (assuming you didn't already have Termux running). Here's a video demonstration:

Show video

Of course, no one should actually have to jump through hoops like this to simply download an image, but here we are…

plwt commented 2 years ago

User feedback: https://support.mozilla.org/en-US/questions/1390028

designzwang commented 2 years ago

User feedback: https://support.mozilla.org/en-US/questions/1390028

Once again, this feedback relates to Pixiv. I don't even know this site, so I'd like to point out once more that this is not a problem with some specific sites, but a general problem with media download on http auth protected resources.

xiiyuki commented 1 year ago

Bump. In 2023, I still cannot download Pixiv images. Error occurs on Firefox 108.1.0, Firefox Nightly 110.0a1, and Fennec 108.1.1 Screenshot_2023-01-06-00-29-39-15_984e1414ae90666a90f12ff17ec14a7f

mrxyzl commented 1 year ago

This is one of the reason i cant use firefox on mobile. Such basic feature is bugged and forever unfix

ctrlcctrlv commented 1 year ago

Incredible to me that the upstream reacted by once again lowering the rank of this issue, from 2 to 333: https://bugzilla.mozilla.org/show_bug.cgi?id=1690037#a59549219_430528 .

Embarrassing is one word for it. I'm sure they've much to do, but this issue also prevents me from using Firefox Mobile, indeed.

ctrlcctrlv commented 1 year ago

Comment hidden (advocacy)

:joy:

Well, you tried. Mozilla doesn't care it is distributing an insecure mobile browser, it seems. I'm going to make my webservers return an image saying "See mozilla-mobile/fenix#17716" because I have this unintended detection method for this browser, that is to say, it re-sends a request w/o a Referer.

boek commented 1 year ago

Moved to bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=1812797

Change performed by the Move to Bugzilla add-on.

mozilla-mobile / fenix