Change the file extension of a Story Project file to something more subject matter related

[klembot]

Originally reported by: greyelf (Bitbucket: greyelf, GitHub: greyelf)

---

The HTML extension of the Story Project file has caused a number of less experienced Author to confuse in a number of ways:

1. They think that it is the Story HTML file created by the Test and Play options, so they try and store their referenced media file relative to it.

2. They think that it is a Story HTML file so they try to view/run it.

3. They think that the library folder is the place to store the Story HTML file they create using the Publish to File location. which can result in them overwriting the Story Project file.

Changing the project's file extension to something other than HTML (or HTM) would help levitate that confusion.

---

• Bitbucket: https://bitbucket.org/klembot/twinejs/issue/261

[klembot]

Original comment by Chris Klimas (Bitbucket: klembot, GitHub: klembot):

---

Totally true that if there are performance issues with publishing the story on save, it would sink this idea. I'd like to test that to see what it looks like.

As for other projects using these files-- seems like we'd have to look at specific examples. The only difference would be that there would be unrelated text around the element that they parse, and if that's even an issue, it would be simple enough to parse.

[klembot]

Original comment by greyelf (Bitbucket: greyelf, GitHub: greyelf):

---

re: storing the Project Passage Data as a published story

1. This would mean having to extract the Passage Data embed within a copy of the Story Format each time the user wants to work on a project which besides adding a large cost to the "open project" process (and in my mind a bit like having to extract source code from a compiled executable) I can see potential problems if there are any issues with the extraction. eg. the user losing their project.

2. Currently there are people that use the contents of the Project file as a data source for applications (games) built using other frameworks.

[klembot]

Original comment by Chris Klimas (Bitbucket: klembot, GitHub: klembot):

---

I'm not yet convinced this is the best solution to the problem, but I agree that the situation should be improved.

One thing to keep in mind is that we're not only deciding this for individual stories, but also the files generated with the Archive button.

A core axiom of Twine 2 is that there is no difference between source files and published ones. You can import a published Twine 2 story in the editor and it will appear exactly as the original author left it. Introducing a separate filename suffix breaks this. So yes, we are using HTML as data storage.

But, I think you'd argue -- we do have source files now; they're just HTML files that can't be viewed in a browser because they're not bound to a format. Right now I think the most suitable solution is to fix that gap, and have the files in the Stories folder actually be published versions of the story. It sounds like it would also meet user expectations, and it would also solve the problems people are seeing when publishing into the Stories folder -- publishing there would reproduce what the Twine app would do on behalf of the user as they edit.

If users are able to overwrite story files in that folder, that is a bug in itself. Twine should be write-protecting them (see nw/directories.js).

I think a useful lens for this is to think about about what should happen when the user tries to open a file in the Stories directory in their file explorer. In an ideal world, would they see...?

1. An empty browser window
2. A browser window with a message telling them to publish the story in Twine to see content
3. The story in the Twine editor
4. The story in published form
5. An OS dialog saying that there is no app registered for this extension

3) is tempting but it adds several other burdens. We shouldn't make users manually associate file extensions with Twine, so that would need to be handled in the NSIS installer and/or the nw-builder process (believe this is set in the app's .plist in OS X). We would also have to gracefully handle the user trying to open several story files at once. We would have to handle files inside the Stories folder differently from a file living outside it. And finally, it's also possible in Windows and macOS (and perhaps Linux too, I don't know) to print a document from Explorer/Finder. We would need to add something to handle that, too.

It feels like a lot compared to going with option 4.

[klembot]

Original comment by furkle industries (Bitbucket: furkle, GitHub: furkle):

---

Whoops! Yeah, both of you are right, I'd meant a valid HTML element, not document. Which to be fair is somewhat more nebulous as the contents of a file than a valid document. But still parseable and loadable as valid HTML, and as far as I can tell passed without change to the published story.

Anyway, I think I'm coming off as wholly against changing any of the filename, which I'm not. Would there be anything better about adding a known string to the end of the story file, e.g. "testTwineStory.html"?

[klembot]

Original comment by Leon Arnott (Bitbucket: L, GitHub: Unknown):

---

My remarks:

• I find it amusing imagining how one would "levitate" confusion :p
• Changing the file extension to something non-conventional does NOT assert anything about the ability to open it in Notepad++ or Sublime or Vim and select the HTML syntax highlighting rule. Consider, for instance, the proliferation of XML-based "proprietary" file formats. I don't believe we have an onus to
• HTML is not intended as a data storage format, and no one interprets files with .html as data files off the bat. This by itself is a bigger "lie" about the file's content than anything else.
• Technically these files are not "standard" HTML documents, because they lack <!doctype html>, and thus are not permitted to omit the <html>, <head> or <body> tags.

[klembot]

Original comment by furkle industries (Bitbucket: furkle, GitHub: furkle):

---

Ah, my mistake, I misread you! My apologies. But the project HTML is a standard HTML document. It's formatted as HTML, it's valid, and the only reason it doesn't display anything is because it contains solely the tw-storydata element, which is styled by default to be display: none'd. You have a point that it may be confusing, but do you see how insisting that users cannot by default edit that file as what it actually is might be somewhat harmful? Isn't this still the thing kind of thing where the better option would still be to put a warning in the docs or a WARNING.TXT in the folder, given that the alternatives are either a. lying about the data contained therein and hoping one's editor doesn't do something wrong; or b. breaking default compatibility for the file with every code editor?

[klembot]

Original comment by greyelf (Bitbucket: greyelf, GitHub: greyelf):

---

I believe you may be confusing the Story HTML file (the one containing the combined Passage Data and Story Format engine) with the Story Project HTML file (the one containing only the Passage Data, the nw.js Twine application's save file), see point 2 of my original post.

I would argue that most users never directly interact with a Story Project HTML file, and using the default HTML filetype association of most operating systems to open one of these files results in a blank web-browser window because it is not a standard HTML document.

[klembot]

Original comment by furkle industries (Bitbucket: furkle, GitHub: furkle):

---

But it would entirely break filetype associations for most users, and make it difficult to open in the browser, which is a critical functionality. The fact that it's an HTML file helps to make it clearer to users that the Story HTML file is the same as a published copy, and that they don't need a separate .tws file to be able to import published games and inspect them. This seems like more of a case for clearer documentation than a change in extension type for a file that is already exactly what it says it is data-wise.

[scottwwh]

Hey folks - just my two cents since I've only started looking at Twine, but seems like most concerns would be resolved by exporting the file as .xml ?

My thinking is that the file:

• Actually is XML, since no tags other than script|style are valid HTML,
• Would remain editable as text,
• Would display in a browser (and be more informative as nodes will be displayed) given some fixes to the file format, which currently throws an error.

I'd be happy to contribute once I get my bearings in the source code.

[klembot]

2.3.1 now saves all stories in publishable format whenever possible (e.g. if something odd is going on, like a story that has no starting point set), and I haven't seen a performance hit. So I feel that, unless issues emerge with this approach, switching the individual story extension to .xml doesn't have much upside.

For archives, maybe XML. I think we would need to wrap passage contents in CDATA sections at least, so there is some work here. But I haven't really seen complaints about people trying to open archive files in a browser and getting confused.

[scottwwh]

Hey, thanks for the quick reply.

I realize now that I'd taken Story Project to mean Archive which isn't the case. Sorry about adding comments to this issue :)

For archives, I am looking to take the output file and split it at the story level for versioning, so having valid XML will help. There are currently some validation errors (hidden attribute without value, plus CDATA as you mention), so I'll investigate and open a branch for PR if there's an easy way to resolve. Having the archive file display its contents in a browser is purely a convenience, but shouldn't require more work than making valid XML. That said, this would necessarily impact both the export and import of archive files, so I'm very mindful about backwards compatibility. I'll create a new issue to track this separately.

[klembot]

The HTML saved as a story is now playable, so I think a lot of the issues raised here have been addressed.