Tool: agfexport - export miscellaneous data from the game source

ericoporto commented 3 months ago

@ivan-mogilko , ages ago you made the pr #1375, with agfexport but then decided to not go forward.

I am making here a tentative idea, I picked your code from that and from the glvar and autoash tools and put together a new tool idea, that makes the three tools into one, using subcommands.

agfexport - parses the game project file (*.agf), and exports various stuff depending on command.

Usage: agfexport <COMMAND> [<OPTIONS>] <input-game.agf> <out-file> Commands: autoash <file> Generate auto script header glvar <header> <body> Generate global variables scripts script-list Exports ordered list of script modules room-list Exports list of active rooms -h, --help Show help message

ivan-mogilko commented 2 months ago

I agree that this is a good idea. Will test this today.

Another thing that may be planned for the future is exporting dialog scripts, but that would require decoupling dialog converter from agf2dlgasc into something that inputs a dialog script from a file, and possibly establish an extension for dialog scripts (has relevance to #1008).

ericoporto commented 2 months ago

I haven't yet deleted autoash and glvar as standalone tools but they may be removed later.

I can later look into decoupling agf2dlasc into some dl2asd command and a asd2asc converter but I would prefer to work on this in a separate PR.

I have some work in a tool that can use the existing tools to generate a building script that can be run to build a game too.

I also don't know if the agf2dta tool should be standalone or go here too - I think I can make a simple "library"/utils file and then just use the function in a simple command here in agfexport. I have some sketch of it.

ivan-mogilko commented 2 months ago

I also don't know if the agf2dta tool should be standalone or go here too - I think I can make a simple "library"/utils file and then just use the function in a simple command here in agfexport. I have some sketch of it.

agf2dta (i assume this created a main game data file) should be a separate tool, because it does not make a simple export into textual format (this is what agfexport seem to do?), but will have to do a big job, processing all game classes and compiling a data file out of the source. It should also be structured well enough to make it easy to extend the process when the data format changes.

The game data writing functions may be placed close to the already existing data reading functions in respective files (such as main_game_file.cpp)

But besides the tool itself, there will be a question of dependency, because right now using existing classes to store intermediate data after conversion from agf will bring Allegro 4 in, as some may have fields of types like Bitmap or RGB, or methods that reference Bitmaps, and others.

Which means that either extra work has to be done on decoupling this dependency, or duplicate classes and/or functions written.

ericoporto commented 2 months ago

I think I can remove dependency on allegro on imagefile without much work, I will try this later and see where I get to, I noticed that bitmapdata is already decoupled.

ivan-mogilko commented 2 months ago

From my memory the most annoying part are GUI classes which also have a number of "runtime" functions (like Draw).

It's quite possible that the easier path would be to create temporary structs duplicating only design-time field members in a separate header, and use these; then come back to this problem later (there's already an opened ticket regarding separating pure data structs and runtime classes).

ericoporto commented 2 months ago

I don't know if it's the same problem but on AGS Script I have a struct that is data only which I use then for serialization and another struct that extends it (in a different script header) that has the methods that are used. Would it be possible to use a similar approach here?

ivan-mogilko commented 2 months ago

I don't know if it's the same problem but on AGS Script I have a struct that is data only which I use then for serialization and another struct that extends it (in a different script header) that has the methods that are used. Would it be possible to use a similar approach here?

This was sort of the plan, yes. Something like proposed in "Proposal" section here #2058.

ivan-mogilko commented 2 months ago

Anyway, in regards to this PR, I tested it, and overall this works, and is a simple program. The issues I found are mostly cosmetic and organizational.

Code style is mismatching our defined style in some places, i.e.
```
if ( ) {
}
```
vs
```
if ( )
{
}
```
(but also mixing both together sometimes)
The help calls this tool:

AGS game's miscellaneous export tool

I find this may be misleading, or rather not specific enough. "Game" may refer to compiled game too. I think there should be a disambiguation. Perhaps "game project export", or similar.

There seem to be a common phrasing mistake (maybe this classifies as a grammatical mistake) in the command descriptions, where "to" precedes "what/from", like:

This will write to the auto-generated script header from

or

This will print to the command line the list of scripts

In English it's correct to put this other way around, e.g.:

This will write the auto-generated script header to a file This will print the list of scripts to the command line

2 commands out of 4 are printing to stdout by default and require extra option to write into a file. This is inconsistent with the rest of commands, and also with the "export" nature of this tool, imho. Is this intentional, and what is the purpose of such decision?
The "room-list" command prints a pair "roomN.crm:Description" for some reason, instead of just list of files. Is that intentional? What is the exact purpose of this command, how is it supposed to be used? Should not it also print room script filenames? (These are not included into "script-list") On another note, I think the room list should be sorted by Number before printing, for convenience (the AGF::ReadRoomList probably does not do that itself).

EDIT:

Oh, and "script-list" only prints asc filenames, but not ash. I think the purpose and way of using this output should be clarified.

ericoporto commented 2 months ago

I think I managed to fix all mentioned issues.

ivan-mogilko commented 2 months ago

I don't quite understand the use case for exporting separate script and script header files. What is the purpose of these actions?

If one builds the game, they would need following:

list of scripts
list of headers to include in each of these scripts.

ericoporto commented 2 months ago

Yeah, exactly as you mentioned, you need the headers list as you essentially concatenate the contents of them all and then you just build the scripts, not necessarily in order - you can build all in parallel now using the command line compiler.

In ags you currently can't but theoretically by the XML tou could have the scripts in a scripts dir and the headers in a headers dir. The filename is an independent property.

ivan-mogilko commented 2 months ago

I think my question is rather, why have these as separate lists, and not a single list with only script names?

When building we either must assume that each script has a respective header of the same name, and then this list of headers can be generated from a list of script names preceding or equal to current script. Or have explicit "include list" per a script which may contain anything.

So, having 2 separate full lists created by 2 separate commands seems redundant. Or the purpose should be clarified. How is this output is supposed to be used?

ericoporto commented 2 months ago

Nah, don't want to do that, filenames are not case sensitive and module names are, there's enough to go wrong there. They are separate properties in the XML, they have to be separate commands.

header list -> needs to have the autogenerated headers added to the top -> goes into the compiler front end

script list -> goes to a build system.

ivan-mogilko commented 2 months ago

I still don't understand, but alright.

ericoporto commented 2 months ago

Headers list has to somehow make into here: https://github.com/adventuregamestudio/ags/blob/6ce71b446cb7a561dc4fbcec57f71a29547c5b1e/Compiler/main.cpp#L30

The script list goes to your build system

ivan-mogilko commented 2 months ago

I realize now, this tool needs to be expanded further and also export:

list of fonts used by the game (because not all files found in the dir are used, and we need to know exact filename with extension);
list of plugins used by the game (for preparing for distribution);
list of translations?

EDIT:

ah! and also, list of custom subdirectories for packaging.

EDIT2: A note, I'm not certain if there's a sense in exporting separate lists of some things, like game fonts, for example. When packaging a game "library" file you only need to know a list of asset files without knowing what they are. Unless there's an interest in knowing particular types of assets belonging to the game, for some reasons.

ericoporto commented 2 months ago

Perhaps files for packaging can be exported as a list of files for packaging. Note that the agspak can already get such list since it can interpret it as an include file pattern.

Note some files are not specified in agf but are to be packed - they are somewhere in the build task in the Editor, things like ogv files at the root of the directory, speech files, preload.pcx and maybe others.

ericoporto commented 2 months ago

But perhaps agspak can have an additional command where it gets a list of files and tries to package the exact list.

adventuregamestudio / ags

Tool: agfexport - export miscellaneous data from the game source #2515