search

adventuregamestudio / ags

AGS editor and engine source code
Other
708 stars 159 forks source link

AGS 3: replace C arrays in ccScript with std containers #2412

Closed ivan-mogilko closed 6 months ago

ivan-mogilko commented 6 months ago

A variant of #2411 for ags3 branch.

This replaces C arrays in ccScript class, and all the related reallocation code, with std containers: std::vectors and std::strings.

A care had to be taken when fixing parsers, as AGS script's bytecode is based on int32 values for historical reasons. Because of that I had to introduce a helper function codesize_i32() which lets to clearly state the purpose of using "current code size" in assignment to int32 variables.

NOTE: I backported a change (1f38e662fe5aa30a6eda69fd375cc5a359cde23e) that reverted to only doing compiled script writing in C++ code (as opposed to C#), because it's very inconvenient to have two versions of this code. I suppose eventually we will be using standalone compiler(s) instead anyway.

ericoporto commented 6 months ago

Do we need the change to native writing here? The ags3 branch doesn't do parallel building and it's quite slow already - if you change a letter in a header file it will rebuild ALL rooms, sequentially.

ivan-mogilko commented 6 months ago

Very well, if that is a concern, then I will revert to have duplicate code.

But for a proper solution, there has to be parallel compilation of scripts and rooms. Because the end goal is to have standalone compiler which is run outside of the editor. Another thing that may be done is to not save compiled scripts in the game data, but as separate *.o files (as already supported by the engine). In that case there won't be any additional copying of the file contents.

ivan-mogilko commented 6 months ago

By the way, compiled room scripts are already saved in native code (this is mentioned in the commit's description), because they are written as a room block, part of the room file format, only implemented in the C++ code.

It is the script modules that are saved using this (relatively) newer C# code.

ericoporto commented 6 months ago

for a proper solution, there has to be parallel compilation of scripts and rooms

The commit that added parallel compilation also added any order of function import and modified the script VM so I was never sure if this was viable in ags3

ivan-mogilko commented 6 months ago

The commit that added parallel compilation also added any order of function import and modified the script VM so I was never sure if this was viable in ags3

I don't remember precise details about function imports, but there should not be anything restricting parallel compilation in the script logic. Scripts may be compiled in parallel if any random script may be compiled on its own. Or, in other words, if compiling one script requires only including headers, and does not require other scripts. That must be achievable in ags3.

What remains is making certain that compiler can technically do parallel compilations, by not using any unique global states, for example (although that is less a problem if it's run as a separate process).

ivan-mogilko commented 6 months ago

The commit that added parallel compilation also added any order of function import and modified the script VM so I was never sure if this was viable in ags3

I probably found this commit, it was also ported to ags3 branch few years ago: 0f977a56f0b1cf293f711e8380bb9ddaa54d2573

I still cannot remember how this is related to the parallel build though.

ericoporto commented 2 months ago

I am trying to remember why this code was written: https://github.com/adventuregamestudio/ags/compare/506c0143aee7605aa916e95029a13698c3f1c5ee..d4107d7df9b3bd252a519d97c76cd4ad82eeca94

Scripts already have a serialization code that is used by the compilers when running on command line or they need to be updated with this in some way?

ivan-mogilko commented 2 months ago

I am trying to remember why this code was written:

Compiled scripts are written in the middle of game data. If game data is written by managed code in C#, using C# objects (io streams, etc), then the program cannot directly use native code to write scripts along. It had to be either duplicated in managed code as well, or written separately and then have the data copied over. I did the second solution in ags4 branch, because it needed RTTI, and I did not want to bother porting whole RTTI serialization to the managed code: https://github.com/adventuregamestudio/ags/blob/a34c7483a590a1c24af668351374c45c56079d44/Editor/AGS.Native/Scripting.h#L59-L71

Strictly speaking, all this may be avoided by saving compiled scripts as separate assets (engine supports that since 3.6.0 iirc).

I would recommend changing to that if you are making "data writing" tool.