auto remove illegal characters

jingyu9575 / multithreaded-download-manager

Download manager extension for Firefox with multithreading support

Mozilla Public License 2.0

264 stars 36 forks source link

auto remove illegal characters #112

Closed Mike-Reddick closed 4 years ago

Mike-Reddick commented 4 years ago

some times I cant download because of illegal characters, please add setting to auto remove illegal characters

Nokia808 commented 4 years ago

Yes, I agree. For example look for the following pages:

https://archive.org/details/lp_swan-lake-the-complete-ballet_pyotr-ilyich-tchaikovsky-richard-bonynge-n/

Only the 1st track [introduction (Moderato Assai) 5:55] can be downloaded. To download it you should do the following:

run the track
open "create a download task" of add-on | select "Media" | enter MP3 in search | select result of search | Download It will downloaded okay. But retry with other tracks, all will failed due to illegal characters !

Other link: https://archive.org/details/lp_sounds-of-three-quarters-of-a-century_minnesota-orchestra-prokofiev-stravinsky/disc1/01.01.+Love+for+Three+Oranges+Suite%2C+Op.+33a.mp3 Using the same way, only 1st 3 tracks can be downloaded while remaining will fail due to illegal characters !

By the way, your add-on do not grasp audio automatically from Internet archive & this is other issue that I will open a special request about it.

jingyu9575 commented 4 years ago

The invalid characters will be replaced in the next version. : will become -.

Nokia808 commented 4 years ago

Many thanks !

Mike-Reddick commented 4 years ago

thanks

Nokia808 commented 4 years ago

@jingyu9575 Hi. I tested new version & still give me same errors ! It is still not fixed ! Please re-test it on the 2 links that I gave you.

jingyu9575 commented 4 years ago

@Nokia808 works here. Are you using Windows?

Nokia808 commented 4 years ago

@jingyu9575 No, I'm using Fedora Linux 30 X64 bit Cinnamon edition. I enabled the following options in "Advanced" - see the following screenshot:

Screenshot

I retried the same file you downloaded, but on my system it was failed again !

jingyu9575 commented 4 years ago

@Nokia808 As far as I know, Linux has no restrictions on filename except / for path separator and null characters, so I did not enable the fix on Linux. I'll test it.

jingyu9575 commented 4 years ago

This is on openSUSE 15.1:

Screenshot_20200117_120617

The : symbol is allowed by Linux, and the extension won't remove it.

Is your download directory in a NTFS partition? That may impose Windows' restrictions, depending on the driver settings. What happens if you use the built-in download on this file, or manually creating a file with name :?

Nokia808 commented 4 years ago

@jingyu9575 1) my drive is Ext4 (version 1.0) 2) these files can not downloaded by buit-in download (only streaming). Only can by downloaded by download manager, like your extension & here exactly your add-on credit ! 3) I tried to rename one of my .docx file. I renamed it into ? & it worked & opened by LibreOffice. Then I renamed it to : & it worked & opened by LibreOffice okay !

jingyu9575 commented 4 years ago

@Nokia808 I've looked into Firefox source code. Apparently they got some issues with saving the about:memory page, but instead of fixing the links, they just disallowed download filenames with : even on Linux. The openSUSE computer I used has Firefox ESR 68 that has not been updated with this restriction.

The next version of the extension will replace : on all systems.

BTW @Nokia808 you may want to switch to the dark theme to match the browser theme.

Nokia808 commented 4 years ago

@jingyu9575 It is working now ! Many thanks !

mo-han commented 4 years ago

@jingyu9575 Please re-open this issue, similar bug here, but I've no idea which char is illegal at all. The download source has a rectified filename which illegal chars are already removed - yes, the site did that automatically for the visitors. The three URLs in above screenshot are: http://95.211.212.246/archive/61747/be610d508ab05c9bf6584d04b50905b307bccaee/fuw34lt9gwq/0?start=1 http://95.211.212.101/archive/26332/cb54299fe60a84a57c4f5ffeb37b9abf7280cf0d/0awuscl9gwq/0?start=1 http://95.211.212.101/archive/73982/fb099446b7b0e7ea4307600344167126adb0ed50/24b0xg09gwq/0?start=1 These URLs will be invalid after 24 hr so there is no way to download them after that time, but the URLs are good, I can download them without MDM extension.

jingyu9575 commented 4 years ago

@mo-han

Looks like Firefox does not allow consecutive spaces in the filename, for whatever reasons. The latest version will remove them.

mo-han commented 4 years ago

@jingyu9575 Wow, thank you for this quick fix, really glad to know it was not some tough bug. ~BTW, how could you find out that "cosecutive spaces" cause? I didn't see any of them at all...~ Yeah I see those spaces in the 'Content-Disposition' header now, it makes sense.