Video titles (and perhaps descriptions) containing particular unicode/emojis/special characters will break the file move functions

[JoryAnderson]

Windows 10 latest updates Tartube v2.4.370

Hi @axcore. Here are some videos where this happens a lot. Not all videos experience this issue, in particular it seems like this affects videos with certain special characters and emojis in the title or description.

This one doesn't work (notice the abundance of special characters) Here's another video with an emoji in the title/description One more

Here's a Beatles' song with an ample description that will work. There's a hyphen in the title but that doesn't seem to break the thumbnail generation or write/move process. Here's Emoji by Brooklyn Queen. This one works too.

Here are my post-processing and file move/keep settings:

Here is my resulting folders. Notice how The Beatles / Brooklyn Queen song is tidied up, while the emoji ones are not:

This also prevents the thumbnail from displaying on the UI:

I downloaded these songs individually, as opposed to the whole folder. Later I decided to test downloading the whole folder. I cleaned out everything from the folder in File Explorer, then refreshed/checked/downloaded the folder from Tartube, and each time got the same result as downloading them individually. The only thing of note is that the Tartube UI doesn't display the emoji videos as downloaded (or display their size) in the Progress tab:

This also prevents thumbnail conversion to jpg (in addition to file moving it to a subfolder). "Write the video's thumbnail to the same folder" will write the webp as expected.

Originally posted by @JoryAnderson in https://github.com/axcore/tartube/issues/508#issuecomment-1572728860

[JoryAnderson]

I decided to test a less egregious video (no unicode, basic special characters), and I got the same result. Here's the video.. It has no description, the only special characters being"(/)" in the title.

In addition, deleting the video via the context menu in the UI did not delete the video from my filesystem.

[sergivonavi]

@axcore I have the same issue. After downloading a playlist, several videos are not recognized by Tartube, despite the fact that they exist in the destination folder (including their metadata and stuff).

Items have no description, etc.:

All missing videos have the following special characters (one of them or both) in their original titles: / :

After downloading they were changed to similar symbols (checked via fontspace.com) in the video file name:

• / (U+002F, forward slash) was changed to ⧸ (U+29F8);
• : (U+003A, colon) was changed to ： (U+FF1A).

, ( ) & do not lead to this problem.

Tried to download these videos separately (via Add new videos) and almost the same thing happened: this time some metadata (description, duration, date) is shown, but not a thumbnail, comments and size.

Right click on an item -> Special -> Download with name -> inputting a name without / or : fixes the issue. But unfortunately this option is not available for videos in a playlist (Download with name is greyed out).

I also tried renaming and replacing the file manually via Right click on an item -> Show video -> Properties -> Set the file but it doesn't work in a playlist (only works for a separately downloaded video):

And I've noticed the exact same behavior described by @JoryAnderson: https://github.com/axcore/tartube/issues/508#issuecomment-1542880570

Windows 10 tartube-with-ffmpeg-2.4.370-64bit.exe

[axcore]

Ok, there seems to be two completely separate issues here. Let's deal with the forward slash character first, using @JoryAnderson 's video.

yt-dlp is apparently not able to download this video, therefore Tartube can't handle it. Here's the output from a minimal download:

$ yt-dlp https://youtu.be/5vXk9M2RJFU
[youtube] Extracting URL: https://youtu.be/5vXk9M2RJFU
[youtube] 5vXk9M2RJFU: Downloading webpage
[youtube] 5vXk9M2RJFU: Downloading android player API JSON
[youtube] 5vXk9M2RJFU: Downloading MPD manifest
[info] 5vXk9M2RJFU: Downloading 1 format(s): 303+251
[dashsegments] Total fragments: 49
[download] Destination: 1v1 M24 intense fight(⧸) [5vXk9M2RJFU].f303.webm
[download] 100% of  102.07MiB in 00:00:28 at 3.57MiB/s
[dashsegments] Total fragments: 1
[download] Destination: 1v1 M24 intense fight(⧸) [5vXk9M2RJFU].f251.webm
[download] 100% of    3.06MiB in 00:00:00 at 3.48MiB/s
[Merger] Merging formats into "1v1 M24 intense fight(⧸) [5vXk9M2RJFU].webm"
ERROR: Postprocessing: Conversion failed!

Your next step is to mark this video as undownloadable at yt-dlp's issues page, noting that the same forward slash character is causing similar results with other videos.

[axcore]

But unfortunately this option is not available for videos in a playlist (Download with name is greyed out).

There are ways around this; I would check the whole channel, then right-click the problem video and do Download with name.... Then I would press CTRL + A to select all videos, then I would click the problem video to remove it from the selection, then I would right-click the remaining selected videos to download them all.

[axcore]

The videos with emojis are all downloaded without problems on my Linux system. I haven't tested MS Windows yet, but I suspect it's the same issue as this one.

Edit: It does seem to be the same issue. As explained in that thread, I don't know the cause or how to fix it. A workaround is to use yt-dlp's --restrict-filenames option: Edit ? General download options... > Files > Filesystem > Restrict filenames to ASCII characters

[axcore]

Should be fixed in v2.5.0.