Editor: store Dialog scripts outside of Game.agf

[ghost]

Among strange design decisions, Dialog scripts are not stored in separate file(s), but as a huge item group inside Game.agf (xml). If script compilers and other tools will be standalone programs, we must move these scripts outside to their respective file or group of files.

This may be also simply convenient to user, because right now there's no way to edit them outside of editor except for searching for them and hacking XML document.

An important nuance is, that the dialog script is a super-set of AGS script. It supports regular commands, but also has it's own. Therefore standalone script compiler won't be enough to compile them, there also has to be a preprocessor to replace special dialog commands with the proper AGS script commands. (Currently this is done by an embedded converter inside Editor program) UPDATE: see #1269

Manual on Dialog scripts: https://github.com/adventuregamestudio/ags-manual/wiki/DialogScript Dialog scripts preprocessor: https://github.com/adventuregamestudio/ags/blob/ags3/Editor/AGS.Editor/Utils/DialogScriptConverter.cs

Since dialog scripts in their original form are not exactly ags script, they should have different file extension (to be invented).

On another hand, maybe the final workflow could support both dialog scripts in special format and in the ags script format.

---

More detailed points on this task:

1. Storage. Now the dialog scripts are stored in Game.agf XML file, <Dialog><Script> item. Instead this script should be stored in a separate file. We need all item contents minus <![CDATA[ opening and closing tags. The name of the file is N.EXT, where N = Dialog's Name (e.g. dDialog1) and EXT is special extension, yet to be decided. (But proper extension is a least concern for now, as it may be adjusted later, before new version is finalized.)

2. Loading. Right now it seems that dialog scripts are loaded along with the rest of the game data, during XML parsing. This may be replaced with function for loading script files explicitly. Such function will then be called when

   • a Dialog is about to be displayed for editing;
   • Editor wants to compile dialog scripts;

3. Hot reloading. Similar to regular scripts, which are stored as separate files already, editor should be detecting file changes to dialog script files, and reload them if they are changed, asking user if necessary. There's a editor setting that controls this behavior for normal scripts, same should be used for dialog scripts.

4. Saving. Similarily, now Editor saves dialog script with the rest of game data into XML. Probably, to comply with regular scripts, instead they should be saved when

   • Dialog is created (create new file, save script placeholder);
   • Dialog is renamed (rename the file);
   • Dialog panel is closed (asking user if save is wanted, again with respect to editor setting);
   • Game is saved, as an extra step after main game data was saved.

---

UPDATE

A dialog script conversion tool is currently in works (see #1269). For now it reads dialogs from Game.agf, but if this task will have a progress we'll need another variant of same tool that reads from individual files.

[ericoporto]

.asd as the suggested extension name.

[ghost]

.asd as the suggested extension name.

The last letter of asc & ash indicate whether they are script body or header, which may probably cause inconsistency with asd, which is a script body, even though not a regular one. EDIT: OTOH that may be okay, since there is no separate "dialog script header" type.

Another option is .dsc as in "Dialog SCript" (as opposed to ".asc" as in "Ags SCript").

And, of course, .ads as in "Ags Dialog Script" :).

EDIT: actually, are we limited by 3 letters there? I never knew if there is a contemporary Linux convention regarding that, but it's no longer a restriction on Windows.

[ericoporto]

Linux doesn't care for extensions, it uses magic numbers to determine a filetype primarily, so there are no character limits, .agsdialog is fine.

File command is the main implementation of this: https://github.com/file/file

Note that magic numbers are not needed if the file is not binary (like .scm is a binary file) and is instead a text file. If the file is a text file, then extensions are useful sugar to determine a filetype, for an external IDE that needs to know how to color a text file, like Atom when loading .ash and .asc files. But there are no limits for extension or naming of the file (like .gitignore). Files that start with a dot are also considered hidden by file explorers.

Edit: what I actually wanted to say is that having the Dialogs as a separate file would be very helpful because it would make easier for two or more people collaborate on a game and also easier to load in an external software that offers grammar correction. :)

[morganwillcock]

I'd probably prefer .asd so that an editor can be configured to match with as{c,d,h}. Maybe the trickier choice is deciding on the file encoding.

I think this change may actually be quite complicated though, since once the content is outside of the XML file it could be modified independently of it. So unfortunately you can't really just write the text to a file and say the job is done. More likely it would be necessary to define the dialog script as something that the script editor can open to get all of the same reload behaviour and save behaviour as regular script or script header, which might mean abstracting out anything that doesn't now apply to all types of editable scripts, and anything that checks 'all scripts' (like the game writer/compiler) would need to differentiate too.

I think this is worth doing though, but whoever tries just needs to make sure that it is treated no differently to another editable text file.

[ericoporto]

I guess for now it would be the same encoding as the .asc and .ash files uses currently - I forgot which is. I don't remember what currently blocks transition to utf-8 or other encoding and if there's an issue where this is mapped out already.

[ghost]

Encoding seem to be same throughout the project files, it's ANSI (with current user system locale), and while non-unicode supportive, it is at least consistent.

[rofl0r]

It supports regular commands, but also has it's own

you mean a separate set of script functions it can call ?

And, of course, .ads as in "Ags Dialog Script" :).

how about simply "DialogScript.asc" ah...

Since dialog scripts in their original form are not exactly ags script, they should have different file extension (to be invented).

could you explain what's the difference to standard ags script ?

[ghost]

Here's an article on dialog scripts: https://github.com/adventuregamestudio/ags-manual/wiki/Settingupthegame#conversations you need to scroll down a little to see the list of special syntax.

In the older version of AGS (pre 2.72 iirc) dialog scripts were processed separately from main script. Today it is converted to ags script by a special preprocessor. All of the special commands are actually replaced with ags script commands, and dialogs themselves are wrapped in regular ags functions with names matching certain convention. https://github.com/adventuregamestudio/ags/blob/ags3/Editor/AGS.Editor/Utils/DialogScriptConverter.cs

[rofl0r]

ah, thanks. i think for the moment it would be sufficient if the editor saves the current converted dialog script to file with .asc extension (and maybe the original script with .asd or whatever), but in the long run it'd be nice to have a standalone command line tool that can do the conversion.

[jduo]

I can see about picking this one up.

[ivan-mogilko]

@jduo please tell what you are planning to implement when you are ready, so that we could discuss this.

The ticket's text is only an outline of the task, and, as noted in the further comments, there may be other related problems that it did not mention.

[ivan-mogilko]

I wrote some points about what should be done (also adding them to the first post):

1. Storage. Now the dialog scripts are stored in Game.agf XML file, <Dialog><Script> item. Instead this script should be stored in a separate file. We need all item contents minus <![CDATA[ opening and closing tags. The name of the file is N.EXT, where N = Dialog's Name (e.g. dDialog1) and EXT is special extension, yet to be decided. (But proper extension is a least concern for now, as it may be adjusted later, before new version is finalized.)

2. Loading. Right now it seems that dialog scripts are loaded along with the rest of the game data, during XML parsing. This may be replaced with function for loading script files explicitly. Such function will then be called when

   • a Dialog is about to be displayed for editing;
   • Editor wants to compile dialog scripts;

3. Hot reloading. Similar to regular scripts, which are stored as separate files already, editor should be detecting file changes to dialog script files, and reload them if they are changed, asking user if necessary. There's a editor setting that controls this behavior for normal scripts, same should be used for dialog scripts.

4. Saving. Similarily, now Editor saves dialog script with the rest of game data into XML. Probably, to comply with regular scripts, instead they should be saved when

   • Dialog is created (create new file, save script placeholder);
   • Dialog is renamed (rename the file);
   • Dialog panel is closed (asking user if save is wanted, again with respect to editor setting);
   • Game is saved, as an extra step after main game data was saved.

If someone knows other potential issues or have ideas, please ammend.

[ericoporto]

Hey, I think we can remove the label context: script compiler from this issue. We have context: game building which, if needed may be added instead.

[persn]

@jduo Are you working on this?

[ivan-mogilko]

After consideration, I guess this belongs to "ags4" branch. This is not obligatory for building game with command-line tools, as they may easily export these dialog scripts from Game.agf.

If these tools are done earlier, they may be even reused in the context of this task: to generate these dialog script files when importing an older project.

[jduo]

@jduo Are you working on this?

I am not working on this now.

[persn]

Okay, I'll probably take over this issue after new years

[ivan-mogilko]

Note there's a standalone tool for converting dialog scripts to regular scripts in works (#1269). For now the tool is reading dialogs from Game.agf. When this task will have some progress we might have to make another variant of the tool that would read dialogs from individual files.

[ericoporto]

I looked at this and started making a version of this and got it working (edit: unfortunately I messed up something when pushing it and lost all my work wth)... So basically the way to go here I believe it's basically copy and pasting code from Script in AGS Types to Dialog there (and adjust for differences), and picking up things from the Script Component to the Dialog Component, and then figuring out how to migrate things. The Script has a few things that assumes it's always a pair of files, so it's not currently quite easy to think on how to share it with Dialog (imagining something both inherit), but the Script Component may have things that can be picked out to some editable file utility or similar.

I also took a note on the AGS Notes plugin, but it uses a completely different approach - all file operations are in the component.

I fully believe in making a version that works with some "repeated code" and then later figure out how to split things apart. One cool thing of this is perhaps how things are picked a part makes easier to add a generic text editor for file thing that could be borrowed for a functionality like AGS Notes that could be used in the Editor - we can pack arbitrary files nowadays so it would be cool to edit pure text files that are later used in-game as resource right in AGS editor itself.

[ivan-mogilko]

So basically the way to go here I believe it's basically copy and pasting code from Script in AGS Types to Dialog there (and adjust for differences), and picking up things from the Script Component to the Dialog Component, and then figuring out how to migrate things. The Script has a few things that assumes it's always a pair of files, so it's not currently quite easy to think on how to share it with Dialog (imagining something both inherit)

I haven't done much thought on this before, but I'd like to suggest to investigate following solution: create a separate DialogScript class that works only with the script. Dialog will become a container class which defines dialog properties and options, and has a reference to DialogScript. This is similar to how Room class has a reference to Script object. I think this may make sense, because Dialog Script is saved in one file containing only script, while dialog settings are saved in Game.agf. (Unless you plan to overhaul that too)

Btw, room scripts also don't have headers, yet they use Script class to define them.

[ericoporto]

Uhm, my main thought was to make a PlainText file object and have both Dialog and Script use that and maybe some utility methods in the Editor side to go with it.

My idea was if someone wanted to add edit capabilities to some other plain text format to the editor it could be leveraged by it.

Also, the dialog stuff that is in the xml that is not the cdata (the text file), I was thinking about just leaving in game.agf and if someone later wants to split those too, it's fine, but I don't think I want play with that part - my use-case need is to load the dialog script text file in a different editor that has advanced spell check capabilities and then I would just reload the file in the Editor.

[ivan-mogilko]

Maybe I miss something, but I don't see how my proposal would contradict to what you intend to do. I've been speaking mostly about organizing classes in AGS.Types, and dividing responsibilities between them.

[ericoporto]

You are right it doesn't. I should have clarified that, probably doing both.