search

UltraStar-Deluxe / format

UltraStar Format Specification
https://usdx.eu/format
MIT License
9 stars 2 forks source link

[Spec] Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL #22

Open achimmihca opened 9 months ago

achimmihca commented 9 months ago

Suggestion

Melody Mania uses the #WEBSITE header field to specify a website that should be used in its embedded browser.

This way, it is possible to specify a youtube link for example that can be used as alternative to local audio and video files.

Note that the same approach might also work with other platforms such as Spotify and Apple Music. Thus, I suggest to use WEBSITE instead of platform specific names such as YOUTUBE or similar.

Note further that WEBSITE fits with other header names of the format, e.g. VIDEO, BACKGROUND, and COVER. These header fields describe what the header value points to.

Use case

Extra info/examples/attachments

No response

Baklap4 commented 9 months ago

Would this field contain a link for example: https://www.youtube.com/watch?v=dvsp6rJRJiw ? If yes this can potentially lead to security/malware risks if no whitelist is made Personally i'm more fan of using an enum/typed field since this is self-explainatory, easy to implement and not prone to beforementioned risks

achimmihca commented 9 months ago

Would this field contain a link for example: https://www.youtube.com/watch?v=dvsp6rJRJiw ?

Yes.

this can potentially lead to security/malware risks if no whitelist is made

I would consider this an implementation detail of karaoke players, not part of the spec.

For instance, Melody Mania loads only websites from supported hosts. And a new host can be supported by adding a corresponding JavaScript file. So you have a whitelist mechanism there. But other players may use a different approach.

As I said, I would not consider whitelisting part of the spec.

i'm more fan of using an enum/typed field

New platforms may arrive in the future. Further, communities might want to host their own song server. Thus, I don't think that an enum with a list of supported platforms is sufficient.

Which websites can be used should be a config issue of the player in one way or another. Not part of the spec.

Baklap4 commented 9 months ago

How would this be combined with say for example our current #MP3 header or the newly introduced #AUDIO or in the later future maybe both #AUDIO and #INSTRUMENTAL? Would this field contain multiple values seperated by character or would we have multiple #WEBSITE tags

If we're going to introduce this tag it'll be an optional field aside from the existing mandatory #MP3 field which loads a local file.

Now thinking of it... what prevents us from chucking in a website in the MP3 or VIDEO or COVER fields? If no http is found search local. If the string is found load from web? This way we don't need an extra field

Within performous we can for example specify a NAS through the network share \\NAS\Somefolder and it'll load everything just fine

achimmihca commented 9 months ago

Good question. My goal is to provide a fallback mechanism when local files are not present.

it'll be an optional field aside from the existing mandatory #MP3 field which loads a local file

Yes.

what prevents us from chucking in a website in the MP3 or VIDEO or COVER fields

Nothing. Actually, Melody Mania and UltraStar Play accept an http URL in these fields that points to an audio / video / image file.

However, my goal is to provide a fallback mechanism when local files are not present. Thereby, it would be nice to stay compatible with offline play in existing UltraStar games.

Thus, I argue that a new field is required for this.

But you are right, other header fields could make use of an online fallback as well. So maybe the field should accept a key value list, where the keys are other header fields.

Further, I would expect the fallback to be something online (e.g., a URI, YouTube video ID, or some other identifier). So maybe a better name for the field would be #ONLINE or #CLOUD?

Example:

#ONLINE: audio="https://something.com/audio.ogg",cover="https://something.com/image.png"

I think this would be similar to what people are doing with new txt files on usdb.animux.de. Their goal is also to make the txt file self-sufficient such that other stuff can be fetched when needed. At the moment they use the #VIDEO field for this, which breaks compatibility with local video files. A standardized approach for this would be nice.

marwin89 commented 9 months ago

SUMMARY

So here is a summary for this issue up till now.

We add this tag to version 1.1.0:

#ONLINE: audio=1k8craCGpgs, video=yCC_b5WHLX0, cover=heart-skips-a-beat-57c3df3b913b0.jpg, background=4f9954bd84f67.jpg

The description in the spec (website) could be:

ONLINE

This tag links to different online ressources and webservices. You can link for a preview/reference/embed of a fitting audio or video that matches with GAP, VIDEOGAP, in the txt-file. Use IDs or URLs from the webservices of your choice (for example: Youtube, Spotify, Apple Music, Vimeo etc.). Right now the tag can only be 255 characters long. Allowed keys are: audio=, video=, cover=, background= Allowed values are: ID (prefered), complete url

@bohning : could you help us out here with usdb syncer meta-tags ? That's a lot to document here. You also got advanced features: cropping, rotating and resizing of covers.

Nianna commented 9 months ago

Can't we use several flat tags for this? e.g. ONLINE_AUDIO, ONLINE_VIDEO, ONLINE_COVER

I do not understand why we are suddenly making the tags way more complex to parse and edit instead of keeping a flat, easy-to-understand structure. Instead of #KEY:VALUE we suddenly have

KEY:KEY=VALUE,KEY=VALUE,KEY=VALUE

Baklap4 commented 9 months ago

I can see why it's been done for USDB_Syncer and the #VIDEO tag as the syncer processes it and makes a #KEY:VALUE header which is easy to parse (which i think is good). Introducing multi-key value within the text file without processing after can make things a little bit tricker and more complex

bohning commented 9 months ago

Can't we use several flat tags for this? e.g. ONLINE_AUDIO, ONLINE_VIDEO, ONLINE_COVER

I do not understand why we are suddenly making the tags way more complex to parse and edit instead of keeping a flat, easy-to-understand structure. Instead of #KEY:VALUE we suddenly have #KEY:KEY=VALUE,KEY=VALUE,KEY=VALUE

Then I would vote for #COVERURL, #VIDEOURLand #AUDIOURL (and sort them right after the respective offline tags), so that

AUDIO:xyz.mp3

AUDIOURL:https://www.youtube.com/watch?v=asdfghjkloi

VIDEO:xyz.mp4

VIDEOURL:https://www.youtube.com/watch?v=asdfghjkloi

COVER:xyz [CO].jpg

COVERURL:https://assets.fanart.tv/xyz.jpg

BACKGROUND:xyz [BG].jpg

BACKGROUNDURL:https://assets.fanart.tv/xyz.jpg

The main reason for using YT and fanart.tv IDs for usdb/usdb syncer is to save characters (max. line length = 255 chars) and to push people towards using those two platforms over others. If the source URLs get their own tag, then at least the first reason vanishes.

achimmihca commented 9 months ago

keeping a flat, easy-to-understand structure

I agree

In any case, I would prefer that we keep the file format consistent. In the flat-key-value scenario, should we also prefer a flat-key-value style for #19 ? For example

#MUSICDBX:VALUEX
#MUSICDBY:VALUEY
...
Nianna commented 9 months ago

should we also prefer a flat-key-value style for https://github.com/UltraStar-Deluxe/format/issues/19

Yes, let's use flat structure for all tags.

bohning commented 9 months ago

We might want/need to add

COVERROTATE:[angle in degrees ccw -> float]

COVERCROP:[left-upper-width-height]

COVERRESIZE:[width --> square: width=height]

if online resources are used. Similarly, for backgrounds (but not rotation needed):

BACKGROUNDCROP:

BACKGROUNDRESIZE:

These are features that the syncer uses to provide square covers and e.g. 16:9 backgrounds (furthermore, it has the option to increase the image contrast for covers --> #COVERCONTRAST:).

marwin89 commented 9 months ago

2nd SUMMARY / VOTE NOW 📈 ✋

Thanks for the discussion. So here is the second summary for this issue. We are close to a final result.

Please vote for this issue till 15th nov 2023 with emoticon 🎉 / 🚀.

Options

🚀 A: Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL and Standard crop/resizing method in USDB SYNCER with central clipping

🎉 B: Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL and #COVERROTATE, #COVERCROP, #COVERRESIZE, #BACKGROUNDCROP, #BACKGROUNDSIZE

👀 C: Add #AUDIOURL, #VIDEOURL, #COVERURL, #BACKGROUNDURL and tags are not flat anymore

❤️ D: Change value structure in #AUDIO/#MP3, #COVER, #VIDEO, #BACKGROUND from flat to complex (for txt-file hostings).

What?

Why?

details A 🚀:

We add to the next version v.1.1.0

#AUDIOURL:https://www.youtube.com/watch?v=asdfghjkloi
#VIDEOURL:https://www.youtube.com/watch?v=asdfghjkloi
#COVERURL:https://assets.fanart.tv/xyz.jpg
#BACKGROUNDURL:https://assets.fanart.tv/xyz.jpg

These tags are optional.

➕ Built-in standard method in USDB SYNCER that crops/resizes cover and background images with centered clipping to max Full-HD. Many CD Covers will not be cropped good.

details B 🎉:

We add to the next version v.1.1.0

#AUDIOURL:https://www.youtube.com/watch?v=asdfghjkloi
#VIDEOURL:https://www.youtube.com/watch?v=asdfghjkloi
#COVERURL:https://assets.fanart.tv/xyz.jpg
#COVERROTATE:-0.3
#COVERCROP:0-200-1000-1000
#COVERRESIZE:1920
#BACKGROUNDURL:https://assets.fanart.tv/xyz.jpg
#BACKGROUNDCROP:10-10-1920-1080
#BACKGROUNDRESIZE:1920-1080

These tags are optional.

details C 👀 :

Introduce complex tags

#AUDIOURL:https=asdfghjkloi
#VIDEOURL:https=asdfghjkloi
#COVERURL:https=asdfghjkloi,resize=500x500,crop=0-200-1000-1000,rotate=-0.1
#BACKGROUNDURL:https=asdfghjkloi,resize=500x500,crop=10-10-1920-1080

details D ❤️ :

We change the value structure in existing # Attributes in the next version v.1.1.0 from flat to complex. (Only if usdb.animux gets an improvement from Alex! Or new USDB.) In the TXT-FILE Hostings the value structure would be 

#AUDIO/#MP3:url
#COVER:url,rotate=,crop=,resize=
#BACKGROUND:url,resize=,crop=
#VIDEO:url,trim=,crop=

Audio/#MP3 stays mandatory, the others stay optional. Games detect files in the specific song folder by file type for instance .mp3, .wav .ogg = Audio, mp4 m4a wmv = VIDEO, Cover = jpg / png file name contains [CO], Background = jpg png file name contains [BG].

The description in the spec (website) could be:

AUDIOURL

This tag links to online ressources and webservices. You can link for a preview/reference/embed of a fitting audio that matches with GAP, VIDEOGAP, in the txt-file. Use complete URLs from the webservices of your choice (for example: Youtube, Vimeo etc.). Right now the tag can only be 255 characters long.

VIDEOURL

This tag links to online ressources and webservices. You can link for a preview/reference/embed of a fitting video that matches with GAP, VIDEOGAP, in the txt-file. Use complete URLs from the webservices of your choice (for example: Youtube, Vimeo etc.). Right now the tag can only be 255 characters long.

COVERURL

This tag links to online ressources and webservices. You can link for a preview/reference/embed of a fitting album art for the song. Use complete URLs from the webservices of your choice (we recommend fanartTV). Right now the tag can only be 255 characters long.

BACKGROUNDURL

This tag links to an online background image file that fits the song. Use complete URLs from the webservices of your choice (we recommend fanartTV). Right now the tag can only be 255 characters long.

COVERROTATE:

Rotates the cover-image from #COVERURL. [angle in degrees ccw -> float]. This is an optional finetuning feature for USDB-SYNCER to make the best out of online ressources.

COVERCROP:

Crops the cover-image from #COVERURL. [left-upper-width-height]. This is a finetuning for USDB-SYNCER. This is an optional finetuning feature for USDB-SYNCER to make the best out of online ressources.

COVERRESIZE:

Resizes the cover-image form #COVERURL. [width --> square: width=height]. This is a finetuning for USDB-SYNCER. This is an optional finetuning feature for USDB-SYNCER to make the best out of online ressources.

BACKGROUNDCROP:

Crops the background image from BACKGROUNDUR [left-upper-width-height]. This is a finetuning for USDB-SYNCER. This is an optional finetuning feature for USDB-SYNCER to make the best out of online ressources.

BACKGROUNDRESIZE:

Resizes the background image from BACKGROUNDURL [width-height]. This is a finetuning for USDB-SYNCER. This is an optional finetuning feature for USDB-SYNCER to make the best out of online ressources.

rakuri255 commented 9 months ago

I do not understand the purpose of these tags. Why should we add a url, which often has no long life? When I look at my Youtube playlist alone, which I created last month, 2 out of 30 videos have already been deleted. These are all tags that can be deprecated in a few months anyway. In my opinion, the txt file should contain all the information offline.

Looks to me like overengineering.

marwin89 commented 9 months ago

@rakuri255 sorry, my summary was missing some explanation.

basisbit commented 8 months ago

I agree with @rakuri255, imho this will not pass the test of time, but instead just cause frustration / confusion for users and unnecessary complexity for devs. If the tags are only for usdb syncer, then you can just as well not have them in the standard. All major implementations of the format support "unknown" tags by ignoring them but not removing them when saving any changes to the file.

achimmihca commented 8 months ago

If the tags are only for usdb syncer, then you can just as well not have them in the standard

I agree. There is no point in adding something to the spec when it is only used by a single tool.

Futher, crop / resize / rotate: this could be specified as part of a BACKGROUNDURL (e.g. as query parameters), so I don't think a dedicated header field is required here.

That said, I still think that a common approach to load stuff from the cloud when not yet present locally would be beneficial to the format.

Please vote for this issue till next saturday

As others pointed out, most people contribute to these projects only occasionally, so a week is probably too short for voting. At least a month feels more appropriate to me.

marwin89 commented 8 months ago
  1. Please, IMO think and discuss more constructive especially about this feature.
  2. Instead of saying no, say how you could imagine how this might work - because I believe you have good ideas and behind that request is a certain need
  3. As a user I always wondered (years ago) where I could store the info about the video I found on vimeo, youtube etc. that fits to the song (gap). as a newcomer I would simply open txt and paste urls in. This is a fact we should use.
  4. in the end this is good for making the txt-files self-sufficient (and independed from one txt-hosting-databases), even if the games don't use it in game play.
  5. these tags (URLS/IDs) are used in usdb-syncer, usdb, the editors - its a general thing
  6. of course youtube links are taken down, but with the work of community in usdb and the use of usdb-syncer they get updated well.
  7. Maybe we can use query parameters in the urls - to reduce the #tag amount. Let's ask bohning.
  8. we miss a big point if we don't include that in the standard. This feature should be official.

Futher, crop / resize / rotate: this could be specified as part of a BACKGROUNDURL (e.g. as query parameters), so I don't think a dedicated header field is required here.

@bohning, is that possible with query paramaters? what do you think about that?

bohning commented 8 months ago

I will think about it and get back to this.

bohning commented 8 months ago

Well, not so easy to sort my thoughts, but I’ll try to give it a shot. I really much like both the idea of a self-contained text file (or, what I think @basisbit recommended in a different issue: self-contained song zip file) as well as the idea of dynamically loading resources, either as a secondary, fallback source or even to avoid having local files all together.

One big problem of usdb was always that most content creators did not bother commenting the audio/video used and that’s where everybody else started getting busy, trying different versions, adapting GAP and BPM, not seldomly without success because the creator used some obscure version and failed to mention that.

With the USDB Syncer and metatags in place, this is a thing of the past and if resources (audio, video, cover, background) get outdated, this can be fixed quite quickly with an active community. So I agree with Marwin that this info should be included in the txt.

If we don’t want to introduce the new tags #AUDIOURL etc., we could go for stating offline and online source in a single tag, e.g.

AUDIO:Journey - Don’t Stop Believin’.m4a|https://www.youtube.com/watch?v=1k8craCGpgs

COVER:Journey - Don’t Stop Believin’ [CO].jpg|https://images.fanart.tv/fanart/dont-stop-believin-5e455e58e7b13.jpg

etc.

That would enable the syncer to download the resources, but also users that do not use the syncer would know the used/recommended resources and karaoke software could be adapted to use either the local or online resources.

Apart from the facts that 1. usdb currently does not support this (#MP3, #COVER, #BACKGROUND are either deleted or autogenerated and overwritten from #ARTIST/#TITLE values) and 2. no karaoke software supports remote resources (UltraStar Play/Melody Mania does for remote files, but not remote video portal URLs such as a YT URL), this could be a way to go.

DoubleDee73 commented 8 months ago

Personally, I don't think it's a good idea to add functionality to the existing tags. I think, it'd be cleaner to have separate tags, that the clients may chose to ignore. I think there's a character limit per tag, so sharing information may be problematic. I also think, that additional tags like crop size, rotation, etc. are over-engineered. These could indeed be added to newly added tags, like

BACKGROUND-URL:https:bla|resize:500x500|rotate:bla

From a dev point of view for Yass, I'm not sure, how to deal with this anyway. Just enabling editing, or also enable downloads. People would have to have additional libraries and stuff for decoding YT, or I would have to bundle it. It'd be more complex in any case...

marwin89 commented 8 months ago

From a dev point of view for Yass, I'm not sure, how to deal with this anyway. Just enabling editing, or also enable downloads.

@DoubleDee73, @Nianna, @rakuri255 don't worry. For EDITOR DEVs it's just enabling editing the #... attributes. ✍️ All download functionallity is done by USDB SYNCER. ⬇️

I also think, that additional tags like crop size, rotation, etc. are over-engineered.

I agree. Yet I see the need behind that. So I suggest following @bohning :

  1. Do not use # attributes for finetuning Co/Bg,
  2. but use a build-in automatic standard cropping/resizing method for covers and backgrounds in USDB SYNCER.
  3. Why? Cause I presume almost all of the cases are done at least good with one method. 1️⃣all
  4. Standard method for covers: cropping and resizing to 1:1 (max 1920px) with centered clipping. ✂️
  5. Standard method for backgrounds: cropping and resizing 16:9 (max Full HD) with centered clipping. ✂️
  6. leave out cover-rotating ❌

BACKGROUND-URL:https:bla|resize:500x500|rotate:bla

Good idea :) I know we had the FLATKEY discussion already, yet I think this feature is so helpful, that we can skip Flatness for that.

I'll update the voting summary (considering your and bohnings thoughts).

codello commented 5 months ago

What is the state of this proposal? If it‘s not too late I‘d like to add the suggestion of using multi-valued tags to solve this issue. This could be similar to how HTTP handles headers:

(The same behavior should of course be applied to other multi-valued tags)

I think this would be a quite elegant solution for this proposal: The fields AUDIO, VIDEO, BACKGROUND, and COVER become multi-valued. The semantics of multiple values in a single field are that different values are to be tried in order. So a header like this:

#AUDIO:foobar.mp3
#AUDIO:file://nas.local/foobar.mp3
#AUDIO:https://youtube.com/…

Would semantically be equivalent to:

#AUDIO:foobar.mp3,file://nas.local/foobar.mp3,https://youtube.com/…

When loading such a song a program would first try to load the local foobar.mp3, then try the file located on a network share and as a final fallback try the YouTube URL (if supported by the client). Alternatively the spec could allow clients to choose a file by their own preference (which would be my preferred solution).

From a specification perspective each value would be a URL. If a value does not include a URL scheme it‘s understood to be a local file located relative to the txt file.

One drawback of this solution is that file names that contain a comma could not be used. I think this is an acceptable limitation but could be solved by implementing an escape sequence of some sort.

DoubleDee73 commented 5 months ago

@codello For now, my implementation is single-valued only, but that doesn't mean, it couldn't be multi-valued. To be honest, personally, I'm not sure, what good it would do, to have multi-values anyway. The games would have to have some sort of fallback strategy, if a file was not found locally, to look into a NAS, or on youtube. The path to the NAS would be very user-specific anyway. I doubt, that would have a shared value.

achimmihca commented 5 months ago

If it‘s not too late I‘d like to add the suggestion of using multi-valued tags to solve this issue

We had the discussion of multi-value vs. single-value (aka. nested vs. flat) tags before. We decided to stay with single-value (flat) tags. For details, see above comments.

codello commented 5 months ago

I think I may have worded that badly.

I read the whole discussion in this issue and I understand the nested vs. flat argument. On that I don't have a strong opinion either way but from what I gathered there seemed to be a tendency towards simpler structures (so flat tags).

To me the multi-value vs. single-value is a different point. The spec already includes multi-valued tags (see #30). I was thinking that maybe the same mechanism could be used to address the need for URL-based files. I'm not suggesting to introduce a complex sub-language for the media files. I just thought that maybe the existing multi-value system might me a good fit as it would alleviate the need for additional tags.

I'll try and phrase my suggestion a bit differently:

I'd like to suggest a solution to this issue where multiple sources for a media file (AUDIO, COVER, etc.) can be defined by specifying the relevant tag multiple times. For example:

...
AUDIO:foobar.mp3
AUDIO:https://youtube.com/...
...

The exact semantics of this would have to be decided. I can think of two plausible approaches:

In addition to this I'd like to unify this syntax of repeating a tag with the existing multi-valued tags in order to reduce the complexity of the spec. But this aspect can be discussed separately.

Baklap4 commented 5 months ago

I'd like to suggest a solution to this issue where multiple sources for a media file (AUDIO, COVER, etc.) can be defined by specifying the relevant tag multiple times. For example:

AUDIO:foobar.mp3
AUDIO:https://youtube.com/...

The web usually handles multiple same keys in this way: ?audio=foobar.mp3&audio=youtube.com/...

Basicly the same as @codello is suggesting. I'd choose @codello answere over any other mentioned solution

achimmihca commented 4 months ago

I like your suggestion in general. However, I am unsure whether this is compatible with old implementations.

The solution with new tags has the advantage that it is backwards compatible because unknown tags are ignored.

codello commented 4 months ago

That's a good point. I don't think it's fundamentally incompatible, but in practice it would be implementation-dependent. There is currently no spec that mandates that headers/tags cannot occur multiple times but I think most implementations assume that's the case.

Specifically USDX should actually be compatible with this suggestion. USDX issues an Error log entry for every MP3 (or VIDEO etc.) entry that does not resolve to a file but if at least one MP3 value actually resolves to an existing file, the song will still load without problems. If multiple MP3 values resolve to files, the last one will be used by USDX.

I don't know how other implementations would deal with repeated headers, but the most straightforward handling would be to just use the last value. So if an implementation doesn't specifically forbid the use of repeated headers this should be backwards compatible.

If we combine this with the multi-valued header syntax used for #TAGS etc. then that part would not be backwards-compatible. So maybe it would be best to defer that part to v2 of the spec which contains breaking changes anyways (or defer even the repeated headers to v2, just to be safe).