Saving state outside Parameter (aka futureBinState, State Stage 1)

[p0nce]

Update 2023:

• [x] Luna implemented stage 1, review the PR and merge. New Dplug major chunk version, that is used in all future DPlug plugins, protected by an Option. => I managed to save/restore UI size in presets (with the caveats it involves).

Remaining for Stage 1:

• [x] Would be nice if we had nothrow parsing functions in dplug.core.binrange, in the wake of WebAssembly and exceptions being difficult => done in Dplug v14

• [x] makeDefaultPreset() is kinda broken since it has no state data. How to deal with that? It is necessary in VST2 I think. (EDIT: perhaps snapshot the default state?)

• [x] Rename getSaveState to a better name, I tend to mixe up the two of them. The opposite of "save" is "load". => stateState after an identifier shoutout 3 vs 0, also is given a reused growable buffer to avoid realloc issues.

• [x] Rename setSaveState to a better name. => loadState, also can return a parse error, constness

• [x] Rename futureBinState to a better name? => nope, good one

  • [x] Use a ref Vec!ubyte for serializeState, it exist already in the context of chunk serialization and will be faster in AAX, with alloc reuse. Else it will allocate non-stop. Document the use of dplug.core.binrange in a DDoc example.
  • [x] Bring LV2 back to feature parity with http://lv2plug.in/ns/ext/state, very probably a single key to a binary chunk to avoids all sorts of LV2 specifics => separated into #808
  • [x] Document setSaveState and getSaveState a bit better with regards to allowed things.
  • [x] Find out why state saving doesn't work in Orion and VSTHost. On load, effSetChunk is not called in Distort. This is quite major issue. It does work in REAPER though. **=> EDIT:
  • [x] Wiki page
  • [x] Changelog entry and link to Wiki page.

• [x] How do people distinguish data that belongs to a preset (such as a wavetable) from data that is global state saved somewhere that is unrelated to a preset (such as window size, dark mode, colorscheme etc...)? Obviously we don't want to change window size each time a factory preset is loaded. => Ha, exact same question in Iplug2 https://github.com/iPlug2/iPlug2/issues/117#issuecomment-880469172 => either a trick or need config file so a VFS abstrction we don't have yet.

  • Would it be OK to just load the global stuff at first chunk read? That way global state can still coexist with preset-stuff => kind of. As a temporary solution, could work. But serializing integer by hand = not fun. Config ideal, since at state loading the UI might not exist yet.

Update 2022:

• [x] need a tree-like, binary serializable structure (CBOR?) (EDIT: not necessarily right now in stage 1) => not done, moved to #786
• [x] gets serialized/deserialized in chunk get/set
• [x] it is something that is stored when the host get/set a chunk
• [x] the host cannot touch it otherwise (so it cannot exist in VST2 without chunk)
• [x] it cannot be automated by the host
• [x] it cannot be set like parameters, by the host
• [x] it needs its own breed of Listener / Observer pattern (EDIT: now unstructured, this is stage 2 to bring some semantics) => not done, moved to #786

Old part of the issue below.

I complained on LV2-devel about lack of chunks.

Meanwhile on IRC:

<ventosus> p0nce: instead of (float-only) lv2:ControlPorts you can use (any-typed) lv2:Parameters
<ventosus> http://lv2plug.in/git/cgit.cgi/lv2.git/tree/plugins/eg-params.lv2
<p0nce> for my complaint about state ?
<p0nce> oh, so you would expose a byte chunk as one such parameter?
<p0nce> is there a defined order in which a LV2 host will restore port values?
<p0nce> (the chunk would have to be last)
<ventosus> yes, possible types are: lv2:Bool/Int/Long/Float/Doble/String/Literal/Chunk/Tuple/Object/Vector/...
<ventosus> p0nce: you're in control which ones you read first/last in your state interface restore callback
<p0nce> interesting, thanks
<ventosus> http://lv2plug.in/doc/html/group__state.html#ac6d423f49c55a6df24ef687450d3cf1a
<p0nce> I thought the state extension was for save/restore of key-value as used in DPF
<ventosus> p0nce: no idea about DPF, sorry
<ventosus> state extension can save/restore *internal* state only ...
<p0nce> I admit I have difficulty knowing what the state extension can do
<ventosus> .. or global parameters (which you register in your ttl) so the host knows about them (and can automate them, etx.)
<ventosus> they're defined like this: http://lv2plug.in/git/cgit.cgi/lv2.git/tree/plugins/eg-params.lv2/params.ttl#n100

[p0nce]

This is the same bug that "VST2 doesn't use chunks", but for LV2 (EDIT from future: no)

[p0nce]

Solution given on the mailing list http://lists.lv2plug.in/pipermail/devel-lv2plug.in/2019-March/001919.html

Asking Modartt would probably help => waiting

[p0nce]

David Robilla makes the very correct point that a plug-in can't upgrade chunks of normalized parameters in an arbitrary way since automation is stored by the host anyway, so there cannot be a correct upgrade in case of a major rework of a plug-in. LV2 conceptions of "meaningful" parameters + internal state is therefore a more fundamental good idea than I expected.

[p0nce]

For Pianoteq, the JUCE LV2 wrapper from FalkTX was used.

[p0nce]

• [x] Transition parameters to being "meaningful" aka not normalized when they can => deferred to #518
• [x] Adopt state like in DISTRO framework, verify if we can build any stateful plugin with this (like, say, a convolver) => well no, DISTROH has string => string maps and this is not really sufficient, need more arbitrary tree-like data instead of flat string[string]

[p0nce]

Internal state could be used for things like: random seed, UI size, general settings.

[p0nce]

I've discussed this with a few plug-in devs and the consensus seems to be that many use their own preset manager, and store the editor dimensions in the regular plug-in state and simply don't save/recall that when saving/loading presets from your preset manager. This means that as discussed, if a user saves/recalls presets via the host the UI will change size.

mmmmmmmmmmmm what do do

[p0nce]

Another solution: https://forum.juce.com/t/problem-storing-and-loading-extra-plugin-info-last-editor-size-etc/50968/4 and JUCE has probably facilities to facilitate savng external config. Interesting.

I'll not that other solution use a chunk abstraction to ease serialization, and we also can do that with dplug.core.binrange.