Formal Specification

[codello]

What does this PR do?

This PR adds a first draft of a more forma specification of the UltraStar file format as suggested in #47.

Closes Issue(s)

Closes #47 Relates to #43, #44, #46, #18 Relates to (closed): #42, #37, #30, #28, #17

Motivation

See #47. A formal specification of the format helps developers make the right decisions when implementing parsers, generators, and other programs interacting with the file format. A formal specification also helps to reduce misunderstandings by explicitly stating what kinds of leeway developers have when creating an implementation.

Additional Notes

This is quite a big PR that has still a lot of points open for discussion. I am trying to highlight these points and potentially remove some paragraphs to be discussed in a later PR. Sections and aspects that are currently under discussion or that have not yet been discussed are marked with a Caution block. Other markers (especially warnings about breaking changes between versions) are intended as part of the spec.

[Baklap4]

Table of contents is auto generated if there are more than 2 headings in a file. You can view it by clicking the hamburger menu: A toc will be shown on the right side

[codello]

The draft should now include all headers except for RELATIVE (which I will add with the appendix).

[codello]

This PR is now ready for review. I realize that this is a very large PR but I'm having trouble removing parts without leaving significant gaps in the spec. I could maybe remove some of the headers and create individual PRs for them. Would that help? I'm happy to split this into multiple smaller PRs (e.g. by section or something) if that helps.

Also I tried to highlight open questions or parts where my wording might change the current spec (I only referenced the USDX implementation). Given that the current state of the spec leaves a lot of room for interpretations it remains to be discussed whether these are actual changes or just formalizations of the status quo.

From a workflow perspective I think it would be beneficial to get some form of a formal specification merged soon or soon-ish. I think this would make it a lot easier to make proposals for new features or changes and discuss the remaining open questions in separate issues. @Baklap4 do you think that would be a good idea that I open those ~15 issues now and link them in the spec before this gets merged? Or should we discuss them one after another?

[bohning]

Kudos! You did a great job here.

[Baklap4]

@Baklap4 do you think that would be a good idea that I open those ~15 issues now and link them in the spec before this gets merged? Or should we discuss them one after another?

I'd discuss them one after another. This gives people time to read up and such :)

As for the review i still gotta get back to your comments. This will probably happen tomorrow 👍

Thanks a lot for the spec it's a lot of effort but much appreciated 🎉

[marwin89]

@codello cool pr, this is high quality! thx for your enthusiasm :) @Baklap4 and @bohning would be nice if you review this. (I could do a review in march).

[codello]

Thank you so much for the review, @Baklap4! I made some changes. Additionally (following the recent discussion on Discord) I clarified that whitespace is ignored around the comma in multi-valued headers.

[codello]

Hey @Baklap4, do you know when you might get a chance to review the latest changes? No pressure, I‘m just eager to discuss the caution blocks and some other ideas for the spec :)

[codello]

Is there anything left to be done before this can get merged? I'm happy to change things to bring this into a mergeable state. For everything else I can open follow-up issues.

[marwin89]

@Baklap4 , can you approve as well? 👋🏻

[codello]

@Baklap4 did you have a chance to take a look at this yet?