Open YuriSizov opened 5 months ago
JD557
commented
5 months ago I know it's a bit off-topic regarding the file format, but if you plan to break the format, please also look into the old issues tracker for issues that are simple to fix, but we're not due to being a breaking change.
The example that immediately comes to mind is https://github.com/TerryCavanagh/boscaceoil/issues/103, which bothers me way more than it should 😅
YuriSizov
commented
5 months ago @JD557 Thanks for a pointer! I definitely want to look into all open issues in the original repo once the bulk of this port is done.
The one you've linked isn't directly related to the file format, but depending on the fix the old format importers must handle it gracefully. So, for instance, if we change existing scales and add "Minor (Compat)" etc at the end of the list, all existing files must seamlessly use that new compat value instead of the original one. Which, I guess, does mean that to properly store new scales we'd need to enforce the next file format version, heh!
nobuyukinyuu
commented
4 months ago Here is a chunk type that can be used for adding metadata to wave format files, for future reference: https://www.recordingblogs.com/wiki/list-chunk-of-a-wave-file
Most players should ignore chunks they don't understand. I seem to remember Cool Edit storing similar metadata in either LIST or INFO chunks. There is no universal standard and other annotation chunk types exist ..
YuriSizov
commented
4 months ago @nobuyukinyuu Ah, that's nice! MIDI also has a few events that might be fitting for song metadata and claim of ownership. This gives more weight to the idea, thanks!
Due to the sequential nature of the saved file format pretty much any change or fix to it requires an introduction of the next format version. This issue is a tracker for ideas, suggestions, and bugs, which require file format changes, so they can be done together as a one version increment.
Ideas
[ ] Fix recorded filter values to support all 32 notes. File format version 3 added support for 32-note long patterns, but it looks like the saver and loader logic was never changed to account for that, and advanced filter settings, per note played, are only handled for the first 16 notes. That logic is hardcoded (i.e. there is no "item amount" value stored to control the read length), so it requires a format change.
[ ] Add support for custom instrument names within the song. Relying solely on the index and the internal instrument name when composing music can be cumbersome, especially if you have two similar (or exactly the same) instruments used with different settings. Colors help only slightly. Custom names can help users identify the purpose of each instrument easier. We can limit the name at, say, 32 characters.
[ ] Add support for song metadata, such as its title and author. While we can't write this to WAV files, probably, it can still be helpful for keeping things in order, especially if you work with other people.
[ ] Implement global filter stacks. Current version of Bosca Ceoil allow only one global filter, although the code suggests that this is an unfortunate limitation due to a bug in SiON or unclear API usage. This can certainly be investigated and fixed, especially given that we control both codebases. If this is done, the file format must be changed to allow a stack of filters instead of just single global one.
[ ] Add a song-wide scale and key setting, so all patterns automatically use that instead of the current default value. You should still be able to change the setting per pattern after the fact, so perhaps a button to quickly reset it should be added as well.
Implementation notes
String support
Right now the document only supports integers as values. In fact, the entire thing is just one line of comma-separated integers (with a trailing comma to wrap things up). Some of the ideas above require us to be able to store strings. The file is a text document, which is great, and using UTF-8 we shouldn't have any encoding issues.
Strings can be stored as two comma-separated values: an int that signifies the string's length, and then the string itself, without any modifications or wrapping, e.g.
The file reader, as it is implemented in Bosca Ceoil Blue, goes through the string character by character. So a string can contain a comma, and it won't break the parser. We would rely solely on length, which is not that different from how instrument and pattern parsing is implemented.