search

openzim / youtube

Create a ZIM file from a Youtube channel/username/playlist
GNU General Public License v3.0
39 stars 26 forks source link

Playlist scraping does not "slug" properly in the ZIM filenames #169

Open kelson42 opened 1 year ago

kelson42 commented 1 year ago

Look at how the ZIM filenames of https://farm.openzim.org/recipes/zenius_id_playlists look like: https://farm.openzim.org/pipeline/46001c89460732f50a144a36

They don't respect the norm, and actually I suspect that if the playlist name has a character which is not supported by the filesystem then it will crash.

rgaudin commented 1 year ago

The recipe is setting the Name via playlists-name using a dynamic variable. This is likely not a very good Name and definitely not convention OK ; but this is a known and tricky problem regarding that specific YT feature

I suppose we could slug whatever value we have when setting the filename (even if a filename was passed) ; in all scrapers (not just this one). that would be safer and mostly harmless.

And yes, we'll probably not encounter much filename issue when creating as we're on ext4 and xfs but users on Windows for instance might have difficulties downloading (I suppose the browser fixes it but I don't really know)

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be now be reviewed manually. Thank you for your contributions.

benoit74 commented 1 month ago

@kelson42 now that we've agreed we will stop to encourage using dynamic metadata and will even stop using it in our Zimfarm, is there really something left to be done here?

kelson42 commented 1 month ago

@benoit74 Not sure myself, but I would recommend to secure that file(name) can be written to the fs in all playlist scenarios.