Should autosaves be versioned?

[curiousdannii]

If you load a storyfile for which you have an autosave from before I changed the stylehints protocol, the stylehints will be turned into CSS that no longer gets applied to the HTML.

While some changes to the autosave format could be handled so that they're backwards compatible (including this one), others might not be. So I'm wondering if the autosave should store a version number, and reject autosaves with an out of date version. It's tricky because there's three levels to consider, the VM, GlkApi, and GlkOte, and any of them could have a change in their autosave data. If an autosave is rejected, it needs to be rejected before any of the autosave state has been resumed. Should there be a function for each level that returns whether it accepts an autosave?

[curiousdannii]

So the current process for doing an autorestore is:

• VM.start() checks for the do_vm_autosave option.
• If found it asks Dialog for an autosave.
• If there's an autosave, then it processes the autosave:
  • First it gives the Glk snapshot to Glk.restore_allstate.
  • Glk.restore_allstate restores everything, and then saves the glkote data for later.
• The VM processes its autosave data, and can now query Glk for existing objects.
• The VM calls Glk.update().
  • Glk.update() includes the stored GlkOte data in its first update.
  • GlkOte.update() can finally process its autosave data.

So the first change that I think needs to happen is that the whole thing needs to be made async. This needs to happen because the Dialog library might be async (for IndexedDB), but it also allows for the possibility that GlkOte is across a network.

In order to allow for versioned autosaves we can make the autorestore functions return a success value. So the process would be changed to:

• VM.start() checks for the do_vm_autosave option.
• If found it asks Dialog for an autosave.
• If there's an autosave, then it processes the autosave:
  • Checks the VM version, stops if out of date.
  • Calls Glk.restore_allstate.
  • Glk checks its version, returns 0 if out of date.
  • Calls GlkOte.restore_autosave
    • GlkOte checks its version, returns 0 if out of date.
    • Saves the data for later.
    • Returns 1.
  • If GlkOte returned 0 then Glk returns 0.
  • Glk restores everything.
  • Returns 1.
  • If Glk returned 0 then don't restore the autosave.
  • The VM processes its autosave data, and can now query Glk for existing objects.
• The VM calls Glk.update().
  • Glk updates GlkOte
  • GlkOte uses its autosave data

The other change I'd like to make is for autosaves to not be JSON object and a RAM array, but instead be a single IFF file. This could be a Quetzal savefile, or just a custom IFF file if the VM doesn't use Quetzal. The VM, Glk, and GlkOte can store their data in their own chunks, so the different layers don't need to be responsible for including other layers data any more. The layers would not be required to use JSON either, if they want to store their data in a more efficient manner.

And for the purpose of #119, the Quetzal savefile format should be adjusted: introduce a new memory chunk XMem, which is the memory xored against the original memory, but without the CMem run length encoding. (Compressing at the end will be more efficient than Quetzal's basic RLE.) And also change the stack chunk to StNE, meaning Stack-Native-Endian, which just means that we don't ensure all the values are in big endian like Quetzal requires. Edit: Hmm, but that would break if there ever was a browser that was big endian. So maybe it should be StLE, stack little endian, and if there was a big endian browser then it would have to convert it.

So all of this involves quite a few changes, which I think should happen all at the same time:

1. Autosaves should use an IFF format
2. If the VM uses Quetzal, change to XMem and StNe chunks
3. Add a version field to each layer's autosave data
4. Change the autorestore API to be async
5. Each layer checks its own version field and returns a success value

@erkyrath Any thoughts on any of this?

[curiousdannii]

Oh, and for #130: making GlkOte display a notice when its autorestoring is easy enough, but it would be even better if it could have a "restart" button too. I haven't thought of a good way yet to send a restart-now signal up through the layers to the VM.

Actually maybe that shouldn't go through the layers, but around them. GlkOte can inform it's creator which can then restart everything. Force restarting will probably require new instances of Glk and GlkOte.

[curiousdannii]

The autosaves should not just store versions, but also which library produced each. I don't have autosaves working in Emglken yet but I would like to. So I'm thinking now it would probably be better to have a separate chunk that stores both the library names and versions for each layer.