write-kodi-descrition option request

f1egmatik commented 7 years ago

Make sure you are using the latest version: run `youtube-dl --version` and ensure your version is 2017.07.02. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[X] I've verified and I assure that I'm running youtube-dl 2017.07.02

Before submitting an issue make sure you have:

[X] At least skimmed through README and most notably FAQ and BUGS sections
[X] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[ ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[X] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Description of your issue, suggested solution and other information

Thank you very much, youtube-dl authors, for this great app! I am ask you create new --write-kodi-description option (additionally to --write-description). If this new option is enabled, yt-dl writes all available info about single video to %filename.nfo and (if downloads playlist) as much as possible info about playlist to tvshow.nfo. So Kodi will take correct (!) descriptions from local (!) drive while scanning. Could you make it?

JimKillock commented 2 years ago

This would be really useful. It would be very helpful to know from the team here (@dirkf ?) how much work it is to create it also. I was about to make the same request, so that is the third time it has been raised, which suggests this is something in demand :)

dirkf commented 2 years ago

Use --write-info-json to generate a filename.info.json file and a tool such as jq to extract from it whatever metadata you want to put into a Kodi .nfo file, in whatever format that is (added: the details are linked in https://github.com/yt-dlp/yt-dlp/issues/3292).

See also PR #30723.

JamesClarke7283 commented 2 years ago

Existing tool which implements something similar is here It converts the info.json file to a .nfo file which is compatible with Kodi.

Best part is there is a config file you can customize to map the youtube Dl fields to the ones Kodi need so you can further expand it.

My opinion on this is, Kodi should implement the ability to read from YoutubeDL info.json files natively, if they already are not working on this.

I have also not had much luck with embedding the metadata in the file or via the xatters. So there are different ways to embed the information existing already and Kodi does not take advantage of them.

Its also worth noting, to get the thumbnails to work on Kodi you rename the [video_name].[image_file_ext] to [video_name]-thumb. [image_file_ext].

So: MrBeast-ChillyChallange.jpg would become MrBeast-ChillyChallange-thumb.jpg so it appears as a thumbnail in Kodi.

Or just call it "thumb.[image_file_ext]"

Relevant docs here: https://kodi.wiki/view/Artwork/Episode

I think they should improve comparability with local metadata formats to resolve this.

dirkf commented 2 years ago

Simply invoke the tool in a --exec ... option.

Generally, it's better to have separate tools for specific tasks that can be combined as needed, even though yt-dl arguably violates that paradigm.

If an integrated solution were desired, ytdl-nfo uses the same licence as yt-dl so it would be possible to subsume it, or better, for Kodi to subsume it. For yt-dl there would be some small changes to match coding conventions.

JamesClarke7283 commented 2 years ago

Thanks @dirkf, i will put the command in once i am done, the .nfo file is trivial to get but the converting of the thumbnail file name, i want to do it without running lots of commands. And so it depends on what thumbnail file extension was used. I just need to get that via the TEMPLATE format but its not listed. Would you happen to know about that?

I know i can run another command most likely to figure that out, but better if its within the ytdl template format.

Also programmatically i am sure this can be done easily as a third party tool as well if they use the above tool to do it. https://github.com/ytdl-org/youtube-dl#adding-support-for-a-new-site=

But a command might be simple enough.

This is what i have so far, it works but not if i change the thumbnail format of course as it doesn't know what file extension it is.

youtube-dl --write-auto-sub  --write-thumbnail --write-info-json -o "%(title)s.%(ext)s" --convert-sub srt \
 --exec after_move:'mv "%(title)s.%(ext)s" "%(title)s-thumb.webp"' \
 --exec after_move:'ytdl-nfo "%(title)s.info.json"' [YOUTUBE VIDEO URL HERE]

Note: I am using a fork of youtube-dl called hypervideo, so it removes any formats which have any patents tied to them(non-free), either in the base format or the optional features in the case of jpeg. So thats why its webp in case you wondered. I know its not a good general rule to do this with forks, but the only difference it makes to is that as far as i know

Ideally i do want to use something like this.

I am sure there is likely a much shorter way of doing it.

 --exec after_move:'mv "%(title)s.%(ext)s" "%(title)s-thumb.%(thumb_ext)s"

Although i know the thumbnail is not part of the video file itself so i see why. There is probably a different "Template" variable i can access, ive been grepping the code and can't find the code behind that format. If i did i might be able to access more of the variables from the commandline.

Of course ideally we would write this using the youtube_dl module, just wanted to see how far i could get with the commandline.

JamesClarke7283 commented 2 years ago

If you want the thumbnails to work in Kodi as well, this is the cleanest one ive been able to do which preserves the formats used

THUMB_EXT=webp youtube-dl --write-subs --write-thumbnail --write-info-json -o "%(title)s.%(ext)s" --convert-sub srt  \
--exec after_move:'mv "%(title)s.$THUMB_EXT" "%(title)s-thumb.$THUMB_EXT"' \
--exec after_move:'ytdl-nfo "%(title)s.info.json"' https://www.youtube.com/watch?v=Ojr6hWdNQUk

Hardcoding it is the only clean solution, but if the thumb nail formats change then you need to adjust it.

The "thumbnail" attribute, gives the full external URL to the thumbnail. If someone has a way to extract just the ".webp" at the end of this, let me know Example is this one: "https://i.ytimg.com/vi_webp/Ojr6hWdNQUk/maxresdefault.webp"

dirkf commented 2 years ago

You appear to be using yt-dlp aliased to youtube-dl. yt-dl doesn't know about after:....

Is there not a filename in the JSON for each thumbnail that has been written?

JamesClarke7283 commented 2 years ago

You appear to be using yt-dlp aliased to youtube-dl. yt-dl doesn't know about after:....

Is there not a filename in the JSON for each thumbnail that has been written?

As far as i know, its youtube-dl as an alias for hypervideo. that is what i am using, which uses youtubedl.

As i explained no, all there was, was this 'thumbnail' attribute.

But i have come close to a solution. I can strip out the rest to get the extension of the file this way.

youtube-dl --write-subs --write-thumbnail --write-info-json -o "%(title)s.%(ext)s" --convert-sub srt \
 --exec after_move:'TE=$(echo "%(thumbnail)s" | sed 's/^.*\.//' ) mv "%(title)s.$TE" "%(title)s-thumb.$TE"' \
 --exec after_move:'ytdl-nfo "%(title)s.info.json"' https://www.youtube.com/watch?v=Ojr6hWdNQUk

I get this

[Exec] Executing command: TE=$(echo "https://i.ytimg.com/vi_webp/Ojr6hWdNQUk/maxresdefault.webp" | sed s/^.*.// ) mv "A New Sitcom Coming This Fall To DTTV....$TE" "A New Sitcom Coming This Fall To DTTV...-thumb.$TE"
mv: cannot stat 'A New Sitcom Coming This Fall To DTTV....': No such file or directory
ERROR: Postprocessing: Command returned error code 1

It does not work, I think i need to simplify it by maybe not even running external commands but builtin bash expression. I just need to get whatever comes after the last ".".

Thats the last part. To get the extension.

dirkf commented 2 years ago

If that's "A fork of youtube-dl without nonfree parts", it's quite strange because

it looks like a fork of a descendant of yt-dl, if not yt-dlp exactly
youtube-dl without nonfree parts == youtube-dl.

Anyhow the command you've used isn't valid for yt-dl, so you should raise the issue with the provider of hypervideo. There seems to be an empty issue tracker at the site linked above, so the owner should have a lot of time to help.

There should be a thumbnails list in the JSON where a filename member might be written in the dict of each thumbnail written. If we don't produce that, it would be a good idea.

JamesClarke7283 commented 2 years ago

If that's "A fork of youtube-dl without nonfree parts", it's quite strange because
1. it looks like a fork of a descendant of yt-dl, if not yt-dlp exactly

2. youtube-dl without nonfree parts == youtube-dl.
Anyhow the command you've used isn't valid for yt-dl, so you should raise the issue with the provider of hypervideo. There seems to be an empty issue tracker at the site linked above, so the owner should have a lot of time to help.

There should be a thumbnails list in the JSON where a filename member might be written in the dict of each thumbnail written. If we don't produce that, it would be a good idea.

Youtube Dl is Free Software. But Youtube-dl allows mp4 and other formats which are patent encumbered, some people want to disable those formats easily(youtubedl's --prefer-free-formats flag i have not found useful to me, its good someone added that as some people find it useful) but the hypervideo fork does by nature(builtin not allow any non-free formats) and hypervideo supports invidious links i think, and i used vanilla youtube-dl a while back and it did not work with invidious links, that last part may have changed IDK,. But that is off-topic. But if you are interested in how software patents can be an issue look here

But, That is outside the scope of making youtube-dl work well with Media Centers like Kodi. This ytdl-nfo command also works for "Plex and Emby, etc" it says so its not just kodi.

Reguarding the thumbnail data issue, i think progromatically it wont be an issue, only on the commandline, because programmatically you can control what the name of the file is firsthand i think. usint the YTDL API.

Anyway this is what i have,

It gives you a list of URL's its fine i can work with that because the file extension is at the end so the "filename" does not matter, as long as i get the file extention right it will work fine.

I found out you can use ${VARNAME##*.} to get the extention of any string in bash, anything after the last fullstop in the string is output, which will give us the extension. I tried it with this command:

So i tried to set an env variable inline and failed with that:

youtube-dl --write-subs --write-thumbnail --write-info-json -o "%(title)s.%(ext)s" --convert-sub srt  \
--exec after_move:'TNAME="%(thumbnail)s" TE=${FNAME##*.} mv "%(title)s."${TE} "%(title)s-thumb."${TE}'  \
--exec after_move:'ytdl-nfo "%(title)s.info.json"' https://www.youtube.com/watch?v=Ojr6hWdNQUk

Result:

[Exec] Executing command: FNAME="https://i.ytimg.com/vi_webp/Ojr6hWdNQUk/maxresdefault.webp" TE=${FNAME##*.} mv "A New Sitcom Coming This Fall To DTTV....$TE" "A New Sitcom Coming This Fall To DTTV...-thumb.$TE"
mv: cannot stat 'A New Sitcom Coming This Fall To DTTV....': No such file or directory
ERROR: Postprocessing: Command returned error code 1

JamesClarke7283 commented 2 years ago

If that's "A fork of youtube-dl without nonfree parts", it's quite strange because
1. it looks like a fork of a descendant of yt-dl, if not yt-dlp exactly

2. youtube-dl without nonfree parts == youtube-dl.
Anyhow the command you've used isn't valid for yt-dl, so you should raise the issue with the provider of hypervideo. There seems to be an empty issue tracker at the site linked above, so the owner should have a lot of time to help.

There should be a thumbnails list in the JSON where a filename member might be written in the dict of each thumbnail written. If we don't produce that, it would be a good idea.

Oh sorry, no the initial issue with the .nfo file was somewhat addressed with this --exec after_move:'ytdl-nfo "%(title)s.info.json"' earlier. The .nfo file is written so as far as that goes, its working when you use that tool. So users have a solution, although not builtin to youtube-dl yet and perhaps could even be improved. I thought i mentioned in my first post, but i may have not expressed it intuitively.

Also, "non-free" parts refers to the patent encumbered audio/video formats which are supported and typically used, it does not mean youtube-dl itself is non-free software(no non-free code running at all in youtube-dl), i can see how that could be misinterpreted. But i left a link in my last post if you were curious about the issue of software patents and how that relates to especially audio formats.

I am just trying to do the thumbnail part to work well. Its the same issue with vanilla youtube-dl, because the thumbnail's default type is "jpg" but there are other types of image formats for thumbnails so i wanted to cover all bases of if a jpg was not available as a file type on that particular video that would still be an issue, this applies to both don't worry about this,

And programmatically the thumbnail can be saved as a kodi compatible name.

Sorry if i confused you on the technical issue i was talking about. I was just hacking here to make a convenient solution for people to use Youtube-dl with Kodi. Because also i was working on setting something up.

dirkf commented 2 years ago

In the yt-dl code the thumbnail filenames are added to the info_dict just after it has been written as JSON. This is probably wrong. This may differ in the fork that you are using.

JamesClarke7283 commented 2 years ago

I just said filenames were there. I was trying to get the file extention from the file names.

JamesClarke7283 commented 2 years ago

@ f1egmatik started this issue @dirkf and i was discussing a separate thing because i though people looking at this issue would want to write thumbnails as well, but i wont talk about that if you dont want me to.

I think there was a misunderstanding, what @f1egmatik was talking about is that there is no .nfo output which is written to be compatable with media centres like Kodi. We can argue wheather you should output to nfo or not.

I was listing a tool which lets you do this, but thought it would likely be easy to integrate the ytdl-nfo functionality. And that my link to that tool people would find useful.

It would effectively create compatibility between youtube-dl and Most media centers not just kodi. So its a very useful feature in my opinion, one i think should remain open in my opinion.

dirkf commented 2 years ago

After 5 years, no-one has wanted this enough to make a PR. Now there is a solution that addresses OP's specific need. And https://github.com/ytdl-org/youtube-dl/issues/13588#issuecomment-1179046387.

[If you think I'm being peremptory in closing feature requests you should see what used to happen, though somehow this one survived. OP's request was for the 2017.07 version.]

JamesClarke7283 commented 2 years ago

I see, so your objection is not that i was talking slightly offtopic about thumbnail(sorry if i was). It was that that feature was never acted on in youtube and you dont think its needed in youtube-dl's end.

I guess that makes sense. But at least now people can find this awesome tool to convert it to nfo format. the ytdl-nfo tool

JamesClarke7283 commented 2 years ago

2017.07

All my commands were run after that version, so don't worry, the commands will apply because those fields were missing in a client i did not even posess. And i ran them in both versions to double check. Only difference was the default thumbnail format. But that was outside of scope, and just something on the side in case someone was running into issues with adapting their downloaded thumbnails to be recognized by kodi.

Thanks for the discussion @dirkf and all your work hard work on this, i appreciate it.

I will hopefully be returning in the future with some pull requests at some point in the future, as there are some features which i would like to use personally that are not open here. But those wont be issues, i will just go ahead and do a PR, lots of issues here already i understand.

Have a great day @dirkf.

And, Happy Hacking (:

ytdl-org / youtube-dl