[2.6] Twee export bugs

[hituro]

Describe the bug.

This is a list of bugs I have found with the Twee export in Twine 2.6

• In the Twee file, there is an extra blank line between each passage
• In StoryData, the format-version is incorrect (it shows the latest installed version of that format, not the one used by the story)
• In StoryData, the IFID is missing if the original story has no IFID, but it's required in Twee3

Steps to reproduce:

Export a story using the 2.6 Twee export

Expected behavior:

The last of those, is an odd situation. I have a story in Twine with no IFID (created in some older version of Twine and brought forward, I think). Exporting it in Twee gives a blank IFID. Importing it still gives a blank IFID. I am not sure if Twine should emit a warning about the missing IFID when exporting to Twee, or a warning when importing from Twee, or generate one when importing.

In any case, Twine has no UI for the user to add an IFID to an existing story, so the warning might not be very helpful even if emitted.

Additional context on this problem.

Also, not so much a bug, but something I think should be an added feature: the StoryData doesn't include the new "tags" attribute on tw-storydata that allows tagging of stories in Twine.

I realise that tags is not a supported attribute in the Twee3 spec, but I think the spec should be updated to include it, now that Twine does.

The importer also ignores this StoryData property, even if provided.

Twine version number

2.6

Does this problem occur with the web version of Twine or the desktop app?

Desktop app

What operating system does this problem occur on?

macOS

If this problem is occurring with the web version of Twine, what browser does it occur on?

None

Presubmission checklist

• [ ] I am interested in working on code that would fix this bug. (This is not required to submit a bug report.)
• [X] I have done a search and believe that an issue does not already exist for this bug in the GitHub repository.
• [X] I have read and agree to abide by this project's Code of Conduct.

[klembot]

In the Twee file, there is an extra blank line between each passage

The Twee 3 spec doesn't have an opinion that I can see about how many lines should come between passages.

In StoryData, the format-version is incorrect (it shows the latest installed version of that format, not the one used by the story)

I can't reproduce this myself. I tried setting a story to Harlowe 2.1.0 and exporting to Twee and the StoryData passage reflected that. Could you attach a story with this problem?

In StoryData, the IFID is missing if the original story has no IFID, but it's required in Twee3

This is perhaps a broader bug. When a story is repaired right now (which happens when you import a story, and every time Twine starts), it checks that the IFID is a string, but doesn't do any further validation. So from Twine's point of view right now, an empty string is valid... which is probably shouldn't treat as such. But this is true across the entire app, not just when it comes to Twee export.

[hituro]

Yes, I very much dithered about putting the IFID issue here, or in a separate bug. The Twine HTML output spec makes it clear that the IFID must be a string of 8-63 characters, so I suppose the file with the blank IFID is invalid as it is.

For the story format, I have a story whose tw-storydata is:

<tw-storydata name="New Red Ruin Gamebook" startnode="1" creator="Twine" creator-version="2.5.1" 
format="Harlowe" format-version="3.2.3" ifid="" options="" tags="Dragon-Warriors" zoom="1" hidden>

(line break added for ease of reading)

Exporting it gives the following StoryData:

:: StoryData
{
  "ifid": "",
  "format": "Harlowe",
  "format-version": "3.3.4",
  "start": "Start",
  "tag-colors": {
    "death": "red",
    "skip": "gray"
  },
  "zoom": 1
}

Note that 3.2.3 is no-longer installed in that particular Twine installation, so I'm guessing the bug happens when the format-version doesn't exist in the app?

[tmedwards]

The Twee 3 spec doesn't have an opinion that I can see about how many lines should come between passages.

I can confirm that it does not. It didn't seem like something we needed to specify.

As an example of how I handled it. Tweego enforces two newlines for readability upon outputting Twee. It also, by default, trims leading and trailing newlines.

[hituro]

@tmedwards I hadn't noticed the COMPILERS: Trailing blank lines must be ignored/omitted. instruction, so I guess it doesn't matter how many lines there are.

@klembot do you want me to edit the bug report to remove that?

[klembot]

Note that 3.2.3 is no-longer installed in that particular Twine installation, so I'm guessing the bug happens when the format-version doesn't exist in the app?

I don't think that this is a bug but I'm open to feedback. This too is not Twee specific. When Twine repairs stories at startup/import, it looks for the closest compatible version of a story format. If your story was Harlowe 2.0.1 and only 2.4.1 (making up numbers) is available, it changes the story format to 2.4.1. The alternative is the story can't be played until the format version is changed, which doesn't seem optimal. Twine only makes safe version changes following semantic versioning, e.g. it will never upgrade a Harlowe 2.0.1 story to 3.0.

[hituro]

@klembot The reason I call it a bug is that the version of the format in the Twee export and the version in the saved story file in "Stories" differ. If you export the story and re-import, you won't get the same result.

However, maybe the bug is that the repair doesn't change the version stored in the file on disk? This might also explain why Twine startup is so slow for me, because it does the same set of repairs every time the app is opened.

[klembot]

I think the current behavior is that it doesn't save changes on startup, because startup in general is a fragile time--Twine is trying to set up in-memory versions of the stories on disk, and trying to save changes in the middle of that process is a bit complex. As soon as you make a change of any kind otherwise, it should save the repaired story.

I doubt that repairs should take very long, but an easy way to verify that is to take the story out of your library folder and see if startup time improves.

So... let's try to restate things because there are a couple issues originally:

1. Twine should repair corrupted IFIDs like empty strings.
2. It would be nice if Twine saved repairs on initial startup.

[hituro]

@klembot I decided to put these questions to a practical test

First, startup. I tried starting up with an empty story folder, and with a folder full of repairable stories, startup time was not affected, so I gracefully concede that suspicion!

Second, looking at a story that was repaired on startup, I could see that the format-version on disk was not changed when the story was opening, but was when a change was made. I think the inconsistency is that you can open a story and Export to Twee, getting the repaired format-version, and then close the story again, without ever affecting the file on disk.

As to saving repairs on startup, I'm not sure if that's right or not, and would leave it to wiser heads than mine. I can see a situation where a story relying on an older format version was inadvertently upgraded to a newer one in the interval between the user opening the app and installing the format version they wanted. Perhaps it would be better to save on story open if a repair was conducted?