Failed to download on line 339

gamebits commented 3 years ago

In my configuration, a "Failed to download" error is consistently returned on line 339, regardless of the feed. Examples below include two feeds confirmed as working in the documentation, as well as my own podcast, where I have more control over the feed for testing purposes:

$ ./podarchive.pl -ev https://atp.fm/rss ./test/
Downloading feed...Failed to download https://atp.fm/rss at ./podarchive.pl line 339.

$ ./podarchive.pl -ev https://www.unmade.fm/episodes?format=rss test
Downloading feed...Failed to download https://www.unmade.fm/episodes?format=rss at ./podarchive.pl line 339.

$ perl ./podarchive.pl https://www.polygamer.net/feed/podcast ./QL/
Downloading feed...Failed to download https://www.polygamer.net/feed/podcast at ./podarchive.pl line 339.

(A --dry-run returns the same error on line 345.)

One exception occurred with another feed from the documentation, though it ultimately failed on the same line:

$ ./podarchive.pl -ev http://www.hellointernet.fm/podcast?format=rss ./QL/
Downloading feed...Done
RSS feed saved at QL/feed.rss
100 episodes
100 - H.I. #136: Dog Bingo
    Writing episode page to QL/100 - H.I. - 136 - Dog Bingo.html
    Downloading http://traffic.libsyn.com/hellointernet/136FinalFinal.mp3 to QL/100 - H.I. - 136 - Dog Bingo - 136FinalFinal.mp3
Fetch failed! HTTP response: 599 [Internal Exception] at ./podarchive.pl line 339.
Command failed:  at ./podarchive.pl line 339.

I'm running bash 3.2.57(1) and perl v5.16.0 on macOS 10.14.6.

janprzy commented 3 years ago

That is very strange, because they all still work for me.

Was this always broken or did it work at some point in the past?
Is it possible for you to test a newer Perl version, e.g. 5.30?
For debugging, please add $File::Fetch::DEBUG=1; after line 5. File::Fetch should then print the backend it is using (e.g. wget or curl) to the command line.

For reference, I am using Perl 5.34 on macOS 11.4, as well as Perl 5.32 on FreeBSD 13.

gamebits commented 3 years ago

Hi! Thank you for the swift response. I've not previously been successful at using this tool, so this is not new behavior for me, unfortunately.

I've now used perlbrew to install perl v5.34.0, which then required I (re-)install XML::RSS::Parser and DateTime::Format::RSS.

Running the command now returns the same error — except this time, it's non-fatal: despite repeated instances of the error, the tool proceeds to download the MP3s. I've tested this with both https://atp.fm/rss and, as seen here, https://www.polygamer.net/feed/podcast:

$ ./podarchive-master/podarchive.pl -ev https://www.polygamer.net/feed/podcast ~/Desktop/pg/
Downloading feed...Command failed:  at ./podarchive-master/podarchive.pl line 339.
Done
RSS feed saved at /Users/kgagne/Desktop/pg/feed.rss
could not find ParserDetails.ini in /Users/kgagne/perl5/perlbrew/perls/perl-5.34.0/lib/site_perl/5.34.0/XML/SAX
116 episodes
116 - Polygamer #114: Dr. Melissa Chaplin on indie game marketing
    Writing episode page to /Users/kgagne/Desktop/pg/116 - Polygamer - 114 - Dr. Melissa Chaplin on indie game marketing.html
    Downloading https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg114-dr_melissa_chaplin.mp3 to /Users/kgagne/Desktop/pg/116 - Polygamer - 114 - Dr. Melissa Chaplin on indie game marketing - pg114-dr_melissa_chaplin.mp3
Command failed:  at ./podarchive-master/podarchive.pl line 339.
115 - Polygamer #113: Maia Weinstock of LEGO Women of Computing
    Writing episode page to /Users/kgagne/Desktop/pg/115 - Polygamer - 113 - Maia Weinstock of LEGO Women of Computing.html
    Downloading https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg113-maia_weinstock.mp3 to /Users/kgagne/Desktop/pg/115 - Polygamer - 113 - Maia Weinstock of LEGO Women of Computing - pg113-maia_weinstock.mp3
Command failed:  at ./podarchive-master/podarchive.pl line 339.
114 - Polygamer #112: Kate Olguin of Other Ocean Interactive
    Writing episode page to /Users/kgagne/Desktop/pg/114 - Polygamer - 112 - Kate Olguin of Other Ocean Interactive.html
    Downloading https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg112-kate_olguin.mp3 to /Users/kgagne/Desktop/pg/114 - Polygamer - 112 - Kate Olguin of Other Ocean Interactive - pg112-kate_olguin.mp3
Command failed:  at ./podarchive-master/podarchive.pl line 339.

et al. The designated destination folder does indeed have the expected RSS, MP3, and HTML files.

So that's an improvement! Perhaps there is an undocumented change or dependency from perl v5.16 to v5.34?

Here is that same command tested with $File::Fetch::DEBUG=1;:

$ ./podarchive.pl -ev https://www.polygamer.net/feed/podcast/ ~/Desktop/PG/
Downloading feed...Running [/usr/local/bin/wget --quiet --passive-ftp --output-document /Users/kgagne/Desktop/PG/file_default https://www.polygamer.net/feed/podcast/]...
Command failed:  at /Users/kgagne/perl5/perlbrew/perls/perl-5.34.0/lib/5.34.0/File/Fetch.pm line 931.
    File::Fetch::_wget_fetch(File::Fetch=HASH(0x7f82053cf0d8), "to", "/Users/kgagne/Desktop/PG/file_default") called at /Users/kgagne/perl5/perlbrew/perls/perl-5.34.0/lib/5.34.0/File/Fetch.pm line 504
    File::Fetch::fetch(File::Fetch=HASH(0x7f82053cf0d8), "to", "/Users/kgagne/Desktop/PG") called at ./podarchive.pl line 340
    main::downloadFile("https://www.polygamer.net/feed/podcast/", "/Users/kgagne/Desktop/PG/feed.rss") called at ./podarchive.pl line 78
 at ./podarchive.pl line 340.
Running [/usr/bin/curl -q --fail --location --output /Users/kgagne/Desktop/PG/file_default https://www.polygamer.net/feed/podcast/]...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  237k    0  237k    0     0  55931      0 --:--:--  0:00:04 --:--:-- 55934
Done
RSS feed saved at /Users/kgagne/Desktop/PG/feed.rss
could not find ParserDetails.ini in /Users/kgagne/perl5/perlbrew/perls/perl-5.34.0/lib/site_perl/5.34.0/XML/SAX
116 episodes
116 - Polygamer #114: Dr. Melissa Chaplin on indie game marketing
    Writing episode page to /Users/kgagne/Desktop/PG/116 - Polygamer - 114 - Dr. Melissa Chaplin on indie game marketing.html
    Downloading https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg114-dr_melissa_chaplin.mp3 to /Users/kgagne/Desktop/PG/116 - Polygamer - 114 - Dr. Melissa Chaplin on indie game marketing - pg114-dr_melissa_chaplin.mp3
Running [/usr/local/bin/wget --quiet --passive-ftp --output-document /Users/kgagne/Desktop/PG/pg114-dr_melissa_chaplin.mp3 https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg114-dr_melissa_chaplin.mp3]...
Command failed:  at /Users/kgagne/perl5/perlbrew/perls/perl-5.34.0/lib/5.34.0/File/Fetch.pm line 931.
    File::Fetch::_wget_fetch(File::Fetch=HASH(0x7f8207ce2228), "to", "/Users/kgagne/Desktop/PG/pg114-dr_melissa_chaplin.mp3") called at /Users/kgagne/perl5/perlbrew/perls/perl-5.34.0/lib/5.34.0/File/Fetch.pm line 504
    File::Fetch::fetch(File::Fetch=HASH(0x7f8207ce2228), "to", "/Users/kgagne/Desktop/PG") called at ./podarchive.pl line 340
    main::downloadFile("https://media.blubrry.com/polygamer/www.polygamer.net/wp-cont"..., "/Users/kgagne/Desktop/PG/116 - Polygamer - 114 - Dr. Melissa "...) called at ./podarchive.pl line 249
 at ./podarchive.pl line 340.
Running [/usr/bin/curl -q --fail --location --output /Users/kgagne/Desktop/PG/pg114-dr_melissa_chaplin.mp3 https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg114-dr_melissa_chaplin.mp3]...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 37.9M  100 37.9M    0     0  4379k      0  0:00:08  0:00:08 --:--:-- 5015k
115 - Polygamer #113: Maia Weinstock of LEGO Women of Computing
    Writing episode page to /Users/kgagne/Desktop/PG/115 - Polygamer - 113 - Maia Weinstock of LEGO Women of Computing.html
    Downloading https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg113-maia_weinstock.mp3 to /Users/kgagne/Desktop/PG/115 - Polygamer - 113 - Maia Weinstock of LEGO Women of Computing - pg113-maia_weinstock.mp3
Running [/usr/local/bin/wget --quiet --passive-ftp --output-document /Users/kgagne/Desktop/PG/pg113-maia_weinstock.mp3 https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg113-maia_weinstock.mp3]...
Command failed:  at /Users/kgagne/perl5/perlbrew/perls/perl-5.34.0/lib/5.34.0/File/Fetch.pm line 931.
    File::Fetch::_wget_fetch(File::Fetch=HASH(0x7f8207d0fca8), "to", "/Users/kgagne/Desktop/PG/pg113-maia_weinstock.mp3") called at /Users/kgagne/perl5/perlbrew/perls/perl-5.34.0/lib/5.34.0/File/Fetch.pm line 504
    File::Fetch::fetch(File::Fetch=HASH(0x7f8207d0fca8), "to", "/Users/kgagne/Desktop/PG") called at ./podarchive.pl line 340
    main::downloadFile("https://media.blubrry.com/polygamer/www.polygamer.net/wp-cont"..., "/Users/kgagne/Desktop/PG/115 - Polygamer - 113 - Maia Weinsto"...) called at ./podarchive.pl line 249
 at ./podarchive.pl line 340.
Running [/usr/bin/curl -q --fail --location --output /Users/kgagne/Desktop/PG/pg113-maia_weinstock.mp3 https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg113-maia_weinstock.mp3]...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 34.6M  100 34.6M    0     0  4935k      0  0:00:07  0:00:07 --:--:-- 5729k
114 - Polygamer #112: Kate Olguin of Other Ocean Interactive
    Writing episode page to /Users/kgagne/Desktop/PG/114 - Polygamer - 112 - Kate Olguin of Other Ocean Interactive.html
    Downloading https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg112-kate_olguin.mp3 to /Users/kgagne/Desktop/PG/114 - Polygamer - 112 - Kate Olguin of Other Ocean Interactive - pg112-kate_olguin.mp3
Running [/usr/local/bin/wget --quiet --passive-ftp --output-document /Users/kgagne/Desktop/PG/pg112-kate_olguin.mp3 https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg112-kate_olguin.mp3]...
Command failed:  at /Users/kgagne/perl5/perlbrew/perls/perl-5.34.0/lib/5.34.0/File/Fetch.pm line 931.
    File::Fetch::_wget_fetch(File::Fetch=HASH(0x7f8206010718), "to", "/Users/kgagne/Desktop/PG/pg112-kate_olguin.mp3") called at /Users/kgagne/perl5/perlbrew/perls/perl-5.34.0/lib/5.34.0/File/Fetch.pm line 504
    File::Fetch::fetch(File::Fetch=HASH(0x7f8206010718), "to", "/Users/kgagne/Desktop/PG") called at ./podarchive.pl line 340
    main::downloadFile("https://media.blubrry.com/polygamer/www.polygamer.net/wp-cont"..., "/Users/kgagne/Desktop/PG/114 - Polygamer - 112 - Kate Olguin "...) called at ./podarchive.pl line 249
 at ./podarchive.pl line 340.
Running [/usr/bin/curl -q --fail --location --output /Users/kgagne/Desktop/PG/pg112-kate_olguin.mp3 https://media.blubrry.com/polygamer/www.polygamer.net/wp-content/uploads/podcast/pg112-kate_olguin.mp3]...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0 37.1M    0 28543    0     0  39200      0  0:16:34 --:--:--  0:16:34 39200^C

janprzy commented 3 years ago

Great to hear that you found a workaround for now.

Looking at your output, it seems to sometimes use wget and sometimes curl, with only wget leading to error messages. May I ask how you installed wget and which version (wget -V) that is?

I am using wget 1.21.1 installed via homebrew.

gamebits commented 3 years ago

I don't recall when/how I installed wget. 🤔 Here's the version output:

$ wget -V
GNU Wget 1.18 built on darwin15.5.0.

-cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls 
+ntlm +opie -psl +ssl/openssl 

Wgetrc: 
    /usr/local/etc/wgetrc (system)
Locale: 
    /usr/local/share/locale 
Compile: 
    gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/usr/local/etc/wgetrc" 
    -DLOCALEDIR="/usr/local/share/locale" -I. -I../lib -I../lib 
    -I/usr/local/include -I/usr/local/include -DNDEBUG -arch x86_64 -Os 
    -mmacosx-version-min=10.7 
Link: 
    gcc -DNDEBUG -arch x86_64 -Os -mmacosx-version-min=10.7 -arch 
    x86_64 -mmacosx-version-min=10.7 -framework CoreFoundation -liconv 
    -L/usr/local/lib /usr/local/lib/libintl.a -liconv -lc 
    -Wl,-framework -Wl,CoreFoundation /usr/local/lib/libssl.a 
    /usr/local/lib/libcrypto.a -ldl -lz -lidn ftp-opie.o openssl.o 
    http-ntlm.o ../lib/libgnu.a

janprzy / podarchive

Failed to download on line 339 #2