Plain text config file

[Deltafire]

Replace the binary config file with plain text, allowing easier configuration and enabling additional options without requiring complication of the config panels.

Something similar to:

# This is a comment
key = value

Would be a good starting point (for more complex configuration we could look at yaml or tcl).

[Deltafire]

Possible libraries we could use:

libconfig boost/program_options json-parser

TOML configuration file format, using cpptoml

[mbionchi]

Hey, I'd like to jump in on that.

[dwhinham]

Awesome! :smiley:

I've been using YAML at work recently for server config files, and I quite like it - easily parsable and very human-readable/editable. An example config file in YAML could look something like this:

What do you think?

[mbionchi]

I think it's good. Originally I was thinking JSON, but there's probably no need for such complications; YAML sounds like a perfect fit.

[farvardin]

yes, YAML would be great. Binarie config are not easy to use.

[MBeijer]

@Deltafire So have we decided for YAML? Or does simple .ini type settings file work as well?

If this work should be done, we need to be able to support the old style format so we can convert it on the fly, so people don't need to set up their configuration from scratch.

I usually prefer not to add external dependencies, so if there's an easy way to add something like YAML without too much work, I'd prefer that.

I'm willing to take on the task if no one else is arsed enough to do it.

[Shoozza]

If ini files are sufficient for the use case, following ini-parser could of interest: https://github.com/benhoyt/inih

[exelotl]

Bumping to say I'd advise against YAML, it's too clever for its own good. I think ini/cfg should do nicely. TOML might also be good, it's well standardised but has a bunch of features we don't really need.

Whichever solution is picked, it would be nice if milky can update the file while still preserving comments.

[caseyjoy]

As a midway step between a full featured config file, and the mostly binary output format we have now, I'm looking into modifying the current system to save and load the config data as strings.

I figured out you can serialize the data out like this:

VERSION = 66304
BUFFERSIZE = 2048
MIXERVOLUME = 256
MIXERSHIFT = 1
RAMPING = 1
INTERPOLATION = 1
MIXERFREQ = 48000
FORCEPOWEROFTWOBUFFERSIZE = 0
AUDIODRIVER = SDLAudio
PLAYMODEKEEPSETTINGS = 0
PLAYMODE = FASTTRACKER2
PLAYMODE_ADVANCED_ALLOW8xx = 1
PLAYMODE_ADVANCED_ALLOWE8x = 0
PLAYMODE_ADVANCED_PTPITCHLIMIT = 1

by modifying the existing TrackerSettingsDatabase::serialize function, and then likely serialize it back in the same way (still working on that part, though).

Why this will likely work (and why it's possible to dump everything out as strings easily in the first place) is that the PPDictionary class it's based on uses PPDictionaryKey, which seems to store everything as a PPString, and then expects you to convert data back to what it's supposed to be later from getStringValue and getIntValue (i.e. BOOL allowExternalKeys = theKey->getIntValue();)

The lucky part about that though, is it means that when serializing everything back in, it should probably be possible to put the string from the file back in, and then have everything keep converting it out again as expected.

Some downsides of a minimal modification approach:

• Comments are a bit tricky, although it might be possible to keep track of them possibly in the Dictionary as another field or elsewhere, and then write them back out first before the actual line (This would also keep modified comments intact, which would be nice).
• Booleans are currently just rendered out as 0/1, since nothing is keeping track of what kind it is, and that's what they are internally. This could probably be addressed by keeping track of what base kinds of variables things are supposed to be in the Dictionary, and then changing what they're rendered out as
• Since the system is converting lists of data to strings so they can do in the database, and then converting them back out later, you get some lines like this:

  SAMPLEEDITORLASTVALUES = newSampleSize=-12345678;changeSignIgnoreBits=-12345678;boostSampleVolume=-885235378;fadeSampleVolumeStart=-885235378;fadeSampleVolumeEnd=-885235378;DCOffset=-885235378;silenceSize=-12345678;waveFormVolume=-885235378;waveFormNumPeriods=-885235378;resampleInterpolationType=-12345678;adjustFtAndRelnote=1;adjustSampleOffsetCommands=0
  PREDEFCOLORPALETTE_0 = 2BFFFFFF80E0FF80FF80FF80E0FFE0808080FF602040A01830406080FFFFFFC0C0C00000008080FF282849103060FFFF00FFFFFFFFFF80202020101010203040FF0000FF0000FF0000FFFF803B00403B00403B00403B00403B00403B00403B00403B00403B00403B00403B00403B00403B00403B00403B00403B00403B00403B0040

  This one is a bit trickier to change, since currently everything expects those to end up like that, so there'd be more changes involved (and might be better suited to a library that already has features for parsing in lists/etc)

• Putting types of config into sections / naming with multiple variables inside another one: Since none of the settings are grouped together normally, there'd probably be a lot of changes involved in getting that working, and libraries would handle the parsing / output better on the config file side

[caseyjoy]

It's looking like this approach works, at least so far - the data will save and load now, and it appears to also load the old style of data. I added a 0 size dword at the beginning in the new format to make it clear to 1.03 that it should treat it as empty and replace the data, as otherwise it'd try to parse it if you rolled back to 1.03 from a version using the new style.

Changing the filename to avoid that kind of thing would also be possible, but that would involve modifying how it selects the config filename: System::getConfigFileName() appears to have different behavior on different systems, so if that gets changed it needs to happen for each platform that currently does different things / has different hardcoded filenames - and it'd need to look for the old one if it can't find the new one, in the places it looks for the standard location currently.

It probably also still needs a dialog to show when the config file has errors in it, as if it's intentionally user editable it's more likely there's going to be typos - probably asking whether the user wants to ignore it, or shut down without replacing the file so they can fix it.