Fixed downloads hanging with an infinite loop if the generated filename exceeds the limits of the filesystem

[RandyScouseGit]

Hi Since November 2023, I haven't been able to complete any downloads from this podcast: RSS: https://anchor.fm/s/5c9dc5f4/podcast/rss Homepage: https://anchor.fm/learnthai Title: You Too Can Learn Thai

I have the podcast in my Radio Downloader 'subscriptions' list. Each Episode pops up in the Downloads list/tab when it is released. Episodes show Status ' Downloading' and the green progress-bar moves all the way over to the right, but it just hangs in that state and the downloaded file never appears in my Windows File Explorer folder. I leave the episode queued for a week or two, but eventually I have to cancel it as other downloads get backed-up/queued behind it.

Windows 10 Home. I Also have the same problem on another machine Windows 11 Home. Latest versions of windows. Radio Downloader ver 0.36-18-7c04ad80 Other feeds/Podcasts are fine, AOK.

Background on the podcast: -The podcast started October 2018, at which point, espisodes were released publicly once a week on the above RSS Feed. I received all of these, no problem. -In Jan 2023 (after ep 126) The provider moved to a 'half and half' system, where one episode was available for free on the previous feed; and the next episode was only available via Patreon or similar paywall. (I don't want the payware ones, I am just trying to get the free ones). After that change in policy, I continued receiving the free ones, no problem. That was January 2023 to late August 2023. -After Aug 2023, ep 163, I started getting the problem described in the first paragraph. I was unable to complete the downloads for eps 166 and 168 (which were on the freeware platform/feed). Somehow I did receive episode 171, 11 Oct 2023. I don't know why this one worked. Since then (mid Oct 2023), I haven't been able to complete the downloads for any episodes. The 'freebie' episodes that should have worked (but didn't) are: ep# Title 176 Wearing shoes in the house, 178 Neighbourhood stores; 181 Waste Sorting 183 Garbage Truck 186 Continuous Learning (Btw, unmentioned episode numbers (177, 179, etc) are on the paywalled stream. I wouldn't expect to receive those. I am not asking about those).

I can 1x stream the problem-episodes from https://podcasts.apple.com/us/podcast/you-too-can-learn-thai-listening-practice-beginner/id1440431373?mt=2 and https://open.spotify.com/show/7LO1UlCxpk2aNV8w5HLQ9Q

but it is tedious stream-ripping them. I would appreciate if you could get them working in Radio Downloader again,

thanks

[ribbons]

Interestingly it doesn't seem like those episodes won't download with Radio Downloader full stop as I just tried the most recent two (using the same Radio Downloader version) and they have completed successfully:

It would surprise me a lot less if the downloads failed with an error (due to the whole essay that seems to have been written in both the podcast episode and description causing some filename or path weirdness) but odd that it just hangs.

Questions and thoughts that spring to mind:

• What's the path you have configured for 'Save downloaded programmes in' (in case that's helpful for me to reproduce the issue)?
• Can you download the enclosure URLs listed in the rss feed successfully via your browser (search for <enclosure and copy and paste the bit between the quotes after URL= into your address bar)?
• If you open the %temp% folder, you should be able to see a randomly named file (with an .m4a extension) created when the download starts, does this disappear before the download hangs?
• Try disabling your antivirus to see if retrying the downloads is then successful

[RandyScouseGit]

Thanks for the reply. To answer your points 1) D:\Downloads\0 Radio DLer DLs 2) Yes 3) No, it stays there until I close Radio Downloader 4) Same symptoms with Windows Defender Realtime protection off

My 'Store' file is 2GB/1200 dls. Guess that doesn't help?

[RandyScouseGit]

edit: I have now 'Cleaned Up' my Downloads list to 50. No change to the original problem.

[RandyScouseGit]

...also tried changing the download folder to D:\Downloads and D:\Documents

[ribbons]

Yes, I wouldn't expect 1200 downloads in the list to cause any particular issues. Nothing is jumping out at me I'm afraid - I suspect the fastest way to tracking down the issue would be to install Visual Studio Community 2017, grab the latest source and see where it is in the code when it hangs - would be much easier to understand if it had the decency to crash instead :smile:

[RandyScouseGit]

If it is working for you, it looks like a problem with my installation. I'm probably just going to remove and reinstall it, unless you particularly want to hand-hold me through loads of diagnostics purely for 'debugging' purposes... ;)

[RandyScouseGit]

So I have now uninstalled Radio downloader (from my second computer (Win11) (which was originally configured by copying the store.db file from the 'main' computer (Win10))).

I was thinking that if I uninstalled everything (including a potentially corrupted store.db file), then that might fix the problem identified in the first post. I used Windows>Apps & Features to uninstall it and restarted the machine afterwards. I confirmed that folder %appdata%\nerdoftheherd.com\Radio Downloader didn't exist after the uninstall.

However, when I reinstalled 0.36, it still had all my Favourites, Subscriptions, etc from the previous installation, which suggests that the original installation wasn't entirely removed, as I intended.

How to I completely remove Radio Downloader, please (so I can do an entirely fresh install)? thanks :)

p.s. (Separately, the reinstallated v0.36 gives an error on launching (and points to a 'Fixed in master branch' webpage, in which the link to the development build (64 bit windows) is a dead link):

[ribbons]

Uninstall doesn't remove store.db (the database is per-user but the application is installed machine-wide) so it's not a surprise that everything was there again after reinstalling. If you want to start from scratch with an empty store.db you can remove or rename it :smile:

Unfortunately the appveyor build artefacts get deleted after about a month these days and I don't commit to the project very often - I've just triggered a rebuild of the most recent commit so if you're lucky the link should work now...

[RandyScouseGit]

Some more testing

On Machine2 (Win11)

• Removing the installed version and starting over with a fresh install (v0.36, production build) and fresh store.db file.
• I can download the more recent episodes of the questioned podcast, no problem, same as you can.
• Changed some config parameters (download folder path and filenaming format, subscribed to the questioned podcast and a couple of others.
• Still OK.
• Backed up store.db file to a USB drive.
• So far, so good.
• The problem starts when I download some older episodes (notably 165, 166, 173 of the questioned podcast). This generates an error and closes the application: It also results in the app thereafter closing itself as soon as it is launched.
• Keeping the store.db and doing a fresh install of the app (i.e temporaily changing the name of store.db to store_temp.db, removing and reinstalling the app, deleting the fresh store.db, then renaming the store_temp.db back to store.db) doesn't help. The app still crashes on launching.
• Replacing store.db with the one I backed up to the USB does fix the problem. The app no longer crashes and I can download recent episodes of the questioned podcast.
• So it looks like downloading those older 'problem' episodes (165, 166, etc) corrupts the store.db. This behaviour is replicable: downloading ep 165 breaks it, replacing store.db with the backup from the USB fixes it.

(This is all on production version v.36. I haven't tried the latest development build, as Windows (Defender) antivirus says the installer (both the 64 bit and the 32bit) has trojan Script/Wacatac.B!ml and deletes it.

Over to Machine 1 (W10). I would expect that using the 'good' store.db from the USB would fix the problem with the hanging downloads, but it doesn't. Recent episodes still hang, as per the OP. Removing and reistalling Radio Downloader on Machine 1 caused me major problems. With a fresh install of Radio Downloader 0.36 (with any and all versions of store.db) causes the app to 'crash' (generate an error and close itself) as soon as it launches. You should have several error reports (31 Jan after 4pm GMT) around this:

I found an older development build (SHA-1 hash 290f48ea23cd1d3df2ca89de3be9341a3ec2240e) I had downloaded (15 Sep 2023). But this gives the same problem on Machine 1 (closes upon launching). Btw, Windows antivirus doesn't have a problem with the installer for that one.

For now, I have used Macrium Reflect to restore Machine 1 back to the state described in the OP. (I also have a sorta-working version on Machine 2, as long as I don't touch those problem episodes, but I'd rather not rely on that).

Can you replicate the problem with podcast eps 165, 166 etc breaking the store.db file ? Maybe that gives a clue...

[ribbons]

(This is all on production version v.36. I haven't tried the latest development build, as Windows (Defender) antivirus says the installer (both the 64 bit and the 32bit) has trojan Script/Wacatac.B!ml and deletes it.

There are definitely bugs related to very long paths in release 0.36 which are fixed in master. I could download them successfully (after lots of tedious warnings about unusual files) using Edge in my Windows 10 VM for what it's worth. VirusTotal doesn't flag up any issues either: https://www.virustotal.com/gui/url/4b6042c5ae372380af887ac55f3f54f6b8d5f8af20f3ec1d22759809badbe0e1

Can you replicate the problem with podcast eps 165, 166 etc breaking the store.db file ? Maybe that gives a clue...

I've just successfully downloaded episodes 166 and 173, however I can't see episode 165 in the feed you gave?

[ribbons]

And out of interest, what kind of drive is your D: drive - internal, external, mapped network drive etc?

[RandyScouseGit]

Thanks for your input.

Sorry, 165 should be 168. I can't read my own handwriting. Drive is internal SDD in both machines.

It does rather look like a length of filename/path issue.

The episodes having (or starting) problems have the longest titles (episodes 166, 168, 173, (151, 153))

I haven't mentioned it so far, but I prefer/change the filename format to: %progname% %longyear%%month%%day% %epname%, which I guess is longer than the default %epname% %day%-%month%-%year%

My latest tests (Machine 2, version 0.36-18-76758C19) downloading (long titled) Episode 168 to DL Folder D:\Users\xyz\Downloads\0 Radio DLer DLs :

Default filename: %epname% %day%-%month%-%year% - downloads OK Shortened filename: %epname% - downloads OK My preferred filename: %progname% %longyear%%month%%day% %epname% - Crashes the app

Regarding the latest development build. I can download the installer OK (via Firefox). It sits in my internet explorer folder. It is only when I try to do anything to it (e.g. rename, copy, run) that Windows (10) jumps-in and deletes (actually, quarantines) it. (Both the appveyor link and uploading Radio_Downloader-win64.msi to virustotal shows it clean - it's just when I touch the file in Windows Explorer that freaks-out Windows Defender). Do you have a slightly older version that you can dump somewhere for me to grab?

[ribbons]

I haven't mentioned it so far, but I prefer/change the filename format to: %progname% %longyear%%month%%day% %epname%, which I guess is longer than the default %epname% %day%-%month%-%year%

Yes, that's relevant - with that filename format I can reproduce the issue, will figure out the exact cause and a fix.

[ribbons]

The underlying issue is that with the file name template %progname% %longyear%%month%%day% %epname% and the bafflingly long podcast and episode names of that particular podcast, the filename that is produced from that template is longer than NTFS supports.

However, as it's clearly wrong for the downloads to hang forever because of it, I've pushed a fix for that to the integration branch (it will now mark the download as errored, with additional details in the info panel) - if you could download and install this build and try downloading those problematic episodes again that would be appreciated.

[ribbons]

Off topic - if you need to add screenshots to GitHub issues in future you can paste them straight in to the input box and the correct markdown will be added automatically. I've taken the liberty of doing this for your image links in the comments for ease of reading and to prevent link-rot in future.

[RandyScouseGit]

Hi Yes, in the new version (0.36-19-e0e8f843), for those 'problem' episodes (166, 168 etc) and 'my' longer filename format, the progress bar goes across and then reports 'Error' in the status column, and this error text bottom left:

---

Date: (episode date) Duration: episode length in minutes Error: Local Problem Encountered an error saving the downloaded file. The filename, directory name. or volume label syntax is incorrect. You may need to select a new location for saving downloaded programmes or adjust the file name format under Options-> Main options.

Just to note that the podcast provider also puts their show notes in that same info field, bottom left. So the error text is appended after about 50 lines of marketing blurb about the Patreon episodes and the transcriptions of the episode content. (So users might never find the detailed error message!)

Do you expect that the integration team will be able to come up a fix (other than just displaying the error messages?) e,g, iferror then truncate the first/last x characters of the filename (or the constituent %progname% or %epname%) and continue to save the downloaded file, but with the modifed filename.

[ribbons]

Yes, in the new version (0.36-19-e0e8f843), for those 'problem' episodes (166, 168 etc) and 'my' longer filename format, the progress bar goes across and then reports 'Error' in the status column, and this error text bottom left:

Great, glad that does the trick for you - thank you for testing and the feedback, I'll push that change to master shortly which should close this issue out.

Do you expect that the integration team will be able to come up a fix (other than just displaying the error messages?) e,g, iferror then truncate the first/last x characters of the filename (or the constituent %progname% or %epname%) and continue to save the downloaded file, but with the modifed filename.

There is no team, just me :smile:. I would be happy to merge a Pull Request which improves the behaviour with over-length filenames in a clean and non-brittle way and works cross-platform (plus includes tests to ensure it doesn't regress in future) but am unlikely to implement it myself as it seems non-trivial at first glance and isn't solving an issue I'm personally encountering.

I think my suggested workaround until then would be to change your file naming template to %progname%\%longyear%%month%%day% %epname% - that way the path of the episode will still contain both the programme and episode names but the filename itself should be valid.

[RandyScouseGit]

I think my suggested workaround until then would be to change your file naming template to %progname%BACKSLASH%longyear%%month%%day% %epname%

Ah, yes thanks. I didn't realise that you could use a backslash (although I have just found the helptext that explicitly says you can!). I don't want to do it globally, but I can take this podcast out of 'Subscriptions' and just throw in a backslash and manually start the download for any future episodes with titles that are too long.
I confirm that this method works (i.e. stops the hanging, completes the download) on the problem files in the old version 0.36-18 (Machine 1, per OP) thanks for your help :)