Make integration of custom opcodes easier

[x87]

CLEO and its successors add many new custom opcodes (see #71 for example). Currently in order to compile the script you have to alter the original *SCM.INI and add new definitions manually. This is error-prone and unfriendly to new users. During Sanny updates those changes could be lost so you need to update the INI again.

Need to come up with a flexible solution that make Sanny aware of custom opcodes without (ideally) touching its files. This affects the compiler, the disassembler, and IDE (Opcode Search).

[MatiDragon-YT]

These are some of the solutions that I would give you if I could

1. In the same way that modes.xml works, you could do something similar to load new opcodes.txt and SASCM.INI with other names, such as opcodeOfCLEO.txt or newOpc.txt from the mode folder.
   • Personally, I don't like XML because of its novice-unfriendly syntax.
2. Write two new files e.g. XXX and ZZZ, for custom opcodes if they don't exist, but incase that these files if they exist, only read files XXX and ZZZ existing.
3. Could add a tool with an interface similar to "User Tools" and thus make it a little more pleasant.

[OrionSR]

Oh wow! Good idea. And is an appropriate issue for the question I had in mind.

Are Restored Opcodes currently supported by Sanny?

Closer to the point, changes to the SCM.ini definitions would influence all of the "command" files, which I've been grouping to include:

• classes.db
• keywords.txt
• opcode.txt (examples.txt)
• scm.ini (opcodes.ini)
• templates.txt (I've been considering renaming some files to be more consistent with the XML definitions, and between all games.) See: #6 - A little feedback would be appreciated.

For keywords in particular, but also CustomVariables.ini, it would nice to be able to have secondary files with duplicate entries that worked in the compiler, but consistently disassembled to the primary example. #20

Thinking ahead to enumerations; I have found it difficult to maintain a long list in constants.txt built around sheets for each group. I would much prefer to manage these groups as individual files. #40

I agree with @MatiDragon-YT 's assessment of the awkwardness of managing the XML mode files. I have given serious consideration to creating a Modes Manager as a learn-to-GUI project with my son, but that's not going to happen any time soon. Neither of us has done this before and, well... he agreed but has shown no interest in starting. I think a better strategy is to allow the user, or ToolPacks, to update files in a manner that doesn't need to modify the XML file.

<opcodes>@sb:\data\vc\opcodes.ini</opcodes>
<opcodes>@sb:\data\vc\plugins\*.opcodes.ini</opcodes>
<enums>@sb:\data\vc\enum\*.enum.txt</enums>

Goal: To update the edit mode the user would only need to place a file with the appropriate naming convention into the proper folder.

[MatiDragon-YT]

Well.. it would not be bad to start like this. in the future, maybe I can generate a simple console program that does that job, that of installing new extensions to XML.

[x87]

A GUI tool that allows to add/remove new entries to modes.xml by just dropping a new file onto the tool window and/or unticking a checkbox to temporarily exclude the file would be really handy.

I agree XML is not the most human-friendly format, yet it's much easier comparing to awkward custom syntax of the classes.db.

[OrionSR]

Before I forget again... I was not able to find an error case for the same keyword text being assigned to different opcode numbers.

I think an Edit Mode Manager tool is an awesome idea worthy of it's own discussion, but doesn't solve the stated goal of:

Need to come up with a flexible solution that make Sanny aware of custom opcodes without (ideally) touching its files.

Suggestion for a Mode Manager progression:

• V1 - A tool for switching the current modes.xml file. User Tools appear to be limited to 9 entries. I'm already using 5 just for this purpose. This version would not need to modify xml files.
• V2 - Ability to copy, paste, disable. delete and sort entire mode definitions. For example. copy a PS2 edit mode from a custom root mode to Sanny's default root mode. Or, duplicate a standard edit mode and update the the title and IDs to support different game paths needed for multiple installations of the same game. -V3 - A full featured Mode Editor. I figure that mode editing will always be an advanced skill. Ideally, a novice would never need to use the editing features.

[x87]

While working on adding CLEO library and its plugins definitions in different edit modes I realized that providing all possible opcodes by default could be misleading and unintentionally make a scripter assume they use an instruction that is available with vanilla game but in fact this is something that requires an extra piece of software. While it's obvious in context of {$CLEO} scripts that you might want to use CLEO opcodes , it's not like that in main.scm or when you use opcodes introduced with some exotic plugins. You should be aware of such dependency on third-party software and explicitly signal Sanny Builder you want to use it.

Hence comes something new in edit modes - the extensions.

What is an extension? It's a named set of files that define new instructions and related language constructs (keywords, classes). Each edit mode could have multiple extensions but it must have at least one which would be used with any script. In other words this is what edit modes currently define using <opcodes>, <keywords> , <examples>, etc. So, say, GTA SA v1.0 mode could now look like:

<mode id="sa" title="GTA SA v1.0" game="sa" type="default">
        <data>@sb:\data\sa\</data>

        <extension name="default">
            <opcodes>@sb:\data\sa\SASCM.ini</opcodes>
            <classes>@sb:\data\sa\classes.db</classes>
            <enums>@sb:\data\sa\enums.txt</enums>
            <keywords>@sb:\data\sa\keywords.txt</keywords>
            <constants>@sb:\data\sa\constants.txt</constants>
            <templates>@sb:\data\sa\templates.txt</templates>
            <examples>@sb:\data\sa\opcodes.txt</examples>
        </extension>

        <variables>@sb:\data\sa\CustomVariables.ini</variables>
        <labels>@sb:\data\sa\CustomLabels.ini</labels>
        <arrays>@sb:\data\sa\CustomArrays.ini</arrays>
        <missions>@sb:\data\sa\missions.txt</missions>
        <text format="sa">@game:\text\american.gxt</text>
        <ide base="@game:\">@game:\data\default.dat</ide>
        <ide>@game:\data\maps\veh_mods\veh_mods.ide</ide>
    </mode>

In addition to the default extension the mode could define custom extensions, e.g. one for standard CLEO opcodes:

<mode id="sa" title="GTA SA v1.0" game="sa" type="default">
        <data>@sb:\data\sa\</data>

        <extension name="default">
            <opcodes>@sb:\data\sa\SASCM.ini</opcodes>
            <classes>@sb:\data\sa\classes.db</classes>
            <enums>@sb:\data\sa\enums.txt</enums>
            <keywords>@sb:\data\sa\keywords.txt</keywords>
            <constants>@sb:\data\sa\constants.txt</constants>
            <templates>@sb:\data\sa\templates.txt</templates>
            <examples>@sb:\data\sa\opcodes.txt</examples>
        </extension>

        <extension name="cleo">
            <opcodes>@sb:\data\sa\cleo4.ini</opcodes>
            <examples>@sb:\data\sa\cleo4.opcodes.txt</examples>
        </extension>

        <variables>@sb:\data\sa\CustomVariables.ini</variables>
        <labels>@sb:\data\sa\CustomLabels.ini</labels>
        <arrays>@sb:\data\sa\CustomArrays.ini</arrays>
        <missions>@sb:\data\sa\missions.txt</missions>
        <text format="sa">@game:\text\american.gxt</text>
        <ide base="@game:\">@game:\data\default.dat</ide>
        <ide>@game:\data\maps\veh_mods\veh_mods.ide</ide>
    </mode>

or any other CLEO plugin:

<mode id="sa" title="GTA SA v1.0" game="sa" type="default">
        <data>@sb:\data\sa\</data>

        <extension name="default">
            <opcodes>@sb:\data\sa\SASCM.ini</opcodes>
            <classes>@sb:\data\sa\classes.db</classes>
            <enums>@sb:\data\sa\enums.txt</enums>
            <keywords>@sb:\data\sa\keywords.txt</keywords>
            <constants>@sb:\data\sa\constants.txt</constants>
            <templates>@sb:\data\sa\templates.txt</templates>
            <examples>@sb:\data\sa\opcodes.txt</examples>
        </extension>

        <extension name="cleo">
            <opcodes>@sb:\data\sa\cleo4.ini</opcodes>
            <examples>@sb:\data\sa\cleo4.opcodes.txt</examples>
        </extension>

        <extension name="cleo+">
            <opcodes>@sb:\data\sa\cleo+.ini</opcodes>
            <examples>@sb:\data\sa\cleo+.opcodes.txt</examples>
        </extension>

        <variables>@sb:\data\sa\CustomVariables.ini</variables>
        <labels>@sb:\data\sa\CustomLabels.ini</labels>
        <arrays>@sb:\data\sa\CustomArrays.ini</arrays>
        <missions>@sb:\data\sa\missions.txt</missions>
        <text format="sa">@game:\text\american.gxt</text>
        <ide base="@game:\">@game:\data\default.dat</ide>
        <ide>@game:\data\maps\veh_mods\veh_mods.ide</ide>
    </mode>

Now the interesting part. By default the IDE and compiler will only use the default extension. This is something that belong to the game opcodes and is always available for the given game. But if you want to use something extra you should add a new line on top of your script: {$use extension_name}. E.g.:

{$use cleo+}
{$use newOpcodes}

Now both the IDE and compiler know your intention and will provide necessary support (autocomplete, highlighting, compiling). The disassembler will be smart enough to restore list of extensions based on which opcodes are actually used in the file.

Using {$CLEO} directive implies {$use cleo} so no changes needed for existing scripts.

The above is only necessary for the scripter. A compiled script will continue depend on the presence of CLEO plugins on user's machine so it's a responsibility of the script author to announce them. However in the future we could integrate some kind of dependency check in the CLEO itself, so it reject scripts if required plugins are missing, to avoid game crashes.

Please let me know your thoughts on this feature.

[x87]

We could also agree on some implicit way of adding extensions, for example, have a subfolder inside edit mode directory named extensions where you create a new folder, say, cleo+ and add new files here:

- Sanny Builder\
-- data\
--- sa\
---- extensions\
----- cleo+\
       |- opcodes.txt
       |- templates.txt
       |- keywords.txt
       |- opcodes.ini

Sanny would automatically register a new extension as if it was defined explicitly in the modes.xml

[x87]

Also have to support conflicting extensions when one opcode number belongs to multiple extensions. Example: clipboard operations plugin & cleo+. The compiler should suggest all possible choices. The disassembler should somehow resolve the correct extension, possibly by analyzing the rest of opcodes used if they belong to the same extension.

[XMDS]

@x87 The function of extended commands is really great.
Just as I replied to you in the forum a few hours ago. I suggest to distinguish between the original OP and the non-original OP. Just as the default original OP uses {$CLEO}, the OP added by the standard CLEO library should be {$CLEO4} and {$CLEO2.0}. This is actually better. In fact, with regard to the OP search bar, whether there is a switch hotkey, switch the search bar of the corresponding type OP according to the extension of the OP.

About the default "keywords"，I noticed that this file is still messy. In fact, the original keyword "if jf wait." and so on appeared for abbreviation, but some OPs about CLEO4 and CLEO+ added in the 3.6.0 keywords are very messy. I mean actually maybe now we are not creating it for shorthand? Then it is necessary to distinguish between several types of OP keywords. As you said, we should create multiple files separately. I also noticed a problem:

In OP.ini and OP.txt: 0A96: $ActorStruct = ped $PLAYER_ACTOR struct 0A97: $CarStruct = vehicle $MyCar struct 0A98: $ObjectStruct = object 0@ struct

Keyword file: 0A96=get_ped_pointer 0A97=get_vehicle_pointer 0A98=get_object_pointer

They are not the same. I think it should be standardized as one kind, just like GTA3sc: 0A96: 0@ = get_ped_pointer 1@ 0A97: 0@ =get_vehicle_pointer 1@ 0A98: 0@ =get_object_pointer 1@

I think the keyword is not necessarily a shorthand for OP, so it can be more standardized. Not only these three OPs, other OPs also have problems.

[x87]

@XMDS Hi, thanks for the input.

OP added by the standard CLEO library should be {$CLEO4} and {$CLEO2.0}.

Can you clarify why do you think it could be better? Those are clearly separated by edit modes itself. You can't use CLEO 2.0 in SA mode and you can't use CLEO4 in III/VC so in both cases {$USE CLEO} is a choice of the standard CLEO opcodes for this platform.

In fact, with regard to the OP search bar, whether there is a switch hotkey, switch the search bar of the corresponding type OP according to the extension of the OP.

I read this as an idea to make a filter in Opcode Search Tool which can enable/disable extensions so you can search in a particular subset of opcodes, say only in CLEO opcodes, right?

In fact, the original keyword "if jf wait." and so on appeared for abbreviation, but some OPs about CLEO4 and CLEO+ added in the 3.6.0 keywords are very messy.

The keywords added to keywords.txt are there and should be kept for backward compatibility. Even if I wanted to change jfto jump_if_false opcode I can only add a secondary keyword and keep them both. I did not want to re-invent CLEO keywords and just picked whichever were used in gta3sc compiler https://github.com/thelink2012/gta3sc/blob/master/config/gta3/cleo.xml They are using full keywords without abbreviations which matches the overall SCR (Rockstar) syntax.

I mean actually maybe now we are not creating it for shorthand? Then it is necessary to distinguish between several types of OP keywords. As you said, we should create multiple files separately.

For 3.7 I'm going to have separate folder for each extension with separate files for opcodes, keywords, examples. CLEO keywords will be extracted from the common keywords.txt to the CLEO extension's keywords.txt.

I think the keyword is not necessarily a shorthand for OP, so it can be more standardized. Not only these three OPs, other OPs also have problems.

I can fix those three. Do you have more examples?

[XMDS]

Can you clarify why do you think it could be better? Those are clearly separated by edit modes itself. You can't use CLEO 2.0 in SA mode and you can't use CLEO4 in III/VC so in both cases {$USE CLEO} is a choice of the standard CLEO opcodes for this platform.

For beginners, there may be a concept of OP, they are not all the game's default OP. I have also noticed that someone is writing the main.scm task script and they do not use the new OP added. So for the OP added by the standard library, maybe it would be better to add the version number. Just like the OP of CLEO4 for beginners, this kind of question will not appear. Of course, there are too many instructions, and now you need to add the corresponding extended instructions for compiling cleo, which is actually very unfriendly now. Among the partners I know about cleo writing groups, 3.6.0 is not so popular. They still use 3.5.0

I read this as an idea to make a filter in Opcode Search Tool which can enable/disable extensions so you can search in a particular subset of opcodes, say only in CLEO opcodes, right?

Yes. This is also good

The keywords added to keywords.txt are there and should be kept for backward compatibility. Even if I wanted to change jfto jump_if_false opcode I can only add a secondary keyword and keep them both. I did not want to re-invent CLEO keywords and just picked whichever were used in gta3sc compiler https://github.com/thelink2012/gta3sc/blob/master/config/gta3/cleo.xml They are using full keywords without abbreviations which matches the overall SCR (Rockstar) syntax.

For 3.7 I'm going to have separate folder for each extension with separate files for opcodes, keywords, examples. CLEO keywords will be extracted from the common keywords.txt to the CLEO extension's keywords.txt.

We don’t need to redefine new keywords, but I just think the current file is very messy. New keywords such as abbreviation (if jf) and cleo4|cleo+ should be divided into multiple files. I very much agree that you use different folders for each type of expansion

I can fix those three. Do you have more examples?

I will sort out some OP's problems in the current keyword file and OP.ini.scm file. I have plenty of time this week. By the way, I'm already writing the material description of Android cleo (including the difference of the csa.csi script. The description of the touch area of the Android screen and the cleo menu). The new OP 0DDX and other OP materials of the Android cleo library. I can submit it to you as a Markdown file before the next weekend, and it can be added to the sb tool help. Tomorrow I will create issues supported by Android CLEO OP

[x87]

Of course, there are too many instructions, and now you need to add the corresponding extended instructions for compiling cleo, which is actually very unfriendly now.

This is why I did so {$CLEO} enables standard CLEO opcodes and only them. If you write a script for the CLEO library you know that your script will only work when the cleo.asi is present. So having {$CLEO} directive allows you to use CLEO opcodes (100 standard opcodes in CLEO 4.4) without any need in {$USE}. If you distribute such a script, you explicitly tell anyone that this is a script for the CLEO library (for example for CLEO 4).

But it does not work this way with opcodes added via CLEO plugins. You may accidentally use an opcode from some plugin without realizing and then it may work on your machine as you have that plugin installed. Now you distribute this script and other people try it, but it does not work as you did not specify that they also need some extra plugin on their machine.

This is why the compiler warns you if you use non-standard opcodes. You have to acknowledge that the opcode belongs to some plugin or a specific game version. It's a one-time change and I think as people get used to it it become a habit. Just like in any other languages there is a standard library provided by the language itself and there are third-party packages developed by someone else. You can't utilize those packages without explicit importing them into your project. The USE directive is essentially an analogue of #include in C++.

[XMDS]

@x87 Today I organized op.txt and op.ini, keyword files, and expansion files. I will upload the newly organized files in a few hours. I have one last question. Shouldn't the OP keyword be created for all OPs? Should we only need to create keywords for the OP added by the CLEO library and CLEO plugin? For example, the keywords of CLEO+ actually do not match the parts in OP.txt and ini. In fact, I don't think it is necessary to create keywords for third-party cleo plugins (such as Cleo+). Or is it better to just specify some nouns as keywords?

[x87]

Eventually we should have keywords and/or classes for all opcodes. But it should be done carefully.

Regarding keywords for third-party plugins. In 3.7 they will move to extensions and wont be part of the standard opcodes unless you add USE directive. It means if you dont have USE CLEO+ you wont see those keywords in Ctrl+Space list, etc.

[XMDS]

Eventually we should have keywords and/or classes for all opcodes. But it should be done carefully.

Regarding keywords for third-party plugins. In 3.7 they will move to extensions and wont be part of the standard opcodes unless you add USE directive. It means if you dont have USE CLEO+ you wont see those keywords in Ctrl+Space list, etc.

as far as I can tell. The OP keyword in the keyword file does not match the OP characters defined in SCM.INI and Opcode.txt. I have cited three examples before. I took the time to modify the keywords of CLEO4/CLEO2.0 to make them unified with SCM.ini and OP.txt. However, the current keywords of CLEO+ are only partially corresponding. If we want to modify it, a large number of OP characters may be changed. This not only means that a lot of time is required, but it is equivalent to rename these OPs. That's why I said that there is no need to create keywords for the OP brought by these CLEO plugins. The reason for the non-correspondence is that you have used the keywords of GTA3sc directly in the sb tool, so this problem occurs. If you want to create keywords for all OPs, there will be more OPs that need to be renamed. Especially time will take up more. I have a good suggestion. We should use some special nouns as keywords instead of the English characters of each OP as keywords. For example, "pointer", "memory", "coordinates", etc. All OPs with these characters will highlight these keywords. This can save a lot of time, specifying special nouns.

[XMDS]

Current CLEO+ keywords:

0D01=rotate_matrix_on_axis
0D02=get_matrix_x_angle
0D03=get_matrix_y_angle
0D04=get_matrix_z_angle
0D0A=get_offset_from_matrix_in_world_coords
0D0F=set_car_model_alpha
0D10=set_char_model_alpha
0D11=set_object_model_alpha

SCM.IN and op.txt:

0D01: rotate_matrix 0@ on_axis 1.0 0.0 0.0 angle 45.0 combine_op 2
0D02: 1@ = matrix 0@ x_angle
0D03: 1@ = matrix 0@ y_angle
0D04: 1@ = matrix 0@ z_angle
0D0A: store_coords_to 1@ 2@ 3@ from_matrix 0@ with_offsets 0.0 2.0 0.0
0D0F: set_car 0@ model_alpha 255 // IF and SET
0D10: set_actor $PLAYER_ACTOR model_alpha 50 // IF and SET
0D11: set_object 0@ model_alpha 50 // IF and SET

@x87 In the current 3.6.0 version. CLEO+ keywords have a lot of non-correspondence, more than the standard library. So we have to rename the OP? This is troublesome. It would be better if only some nouns were specified.

[x87]

@XMDS

0D01=rotate_matrix_on_axis
0D01: rotate_matrix 0@ on_axis 1.0 0.0 0.0 angle 45.0 combine_op 2

I don't see a discrepancy here. A benefit of opcode syntax is that it allows inline descriptions for all parameters whereas a keyword is just a distilled shortcut describing the command as the whole. Are you suggesting to rename the keyword to be 0D01=rotate_matrix_on_axis_angle_combine_op or what?

[x87]

For example, we've been having the keyword jf and opcode 004d: jump_if_false for years and no one called this as a problem.

[XMDS]

@XMDS

0D01=rotate_matrix_on_axis
0D01: rotate_matrix 0@ on_axis 1.0 0.0 0.0 angle 45.0 combine_op 2

I don't see a discrepancy here. A benefit of opcode syntax is that it allows inline descriptions for all parameters whereas a keyword is just a distilled shortcut describing the command as the whole. Are you suggesting to rename the keyword to be 0D01=rotate_matrix_on_axis_angle_combine_op or what?

If the keyword is to simplify the command, it is actually just like 0D01, which is too long to be memorable. What is the meaning of keywords? When we write cleo and use op, we may directly copy the OP in OP.txt, and after decompilation, it will become the name defined by SCM.INI. So the meaning of the keyword itself seems unclear? Because it is not like if if wait and other abbreviations, if it can not be used in writing, then what is the meaning? I believe no one likes to memorize these keywords, they are too long. . . . That’s why I recommend that keywords should be aligned with the complete description of op, or only some nouns should be set as keywords (as in my example "thread, pointer, coordinate)

[XMDS]

0D01: rotate_matrix 0@ on_axis 1.0 0.0 0.0 angle 45.0 
0
0D01=rotate_matrix

For example. It is better to use the descriptive character at the beginning of op itself as a keyword. We can see the highlighted keywords even after decompilation. Even if we search for 0D01 directly from oP.txt and copy it to our script, it will be highlighted. What I mean is this, the Op description characters in SCM.INI and OP.txt should be consistent with the keyword itself. However, currently the keyword text does not support spaces and skip parameters to display keywords. Therefore, only the descriptive characters at the beginning can be used as keywords. Or use the word "matrix" as a keyword and apply to all ops that have the descriptive character "matrix"

[x87]

there are two flavors when we talk about keywords. Keywords could be seen as part of the language itself (like if or while) and also there are keywords that are names of the commands. In the original gta3script script they did not use opcode syntax. Each command had a name. You can read more about it here: https://gtamods.com/wiki/SCM_language

So here is the list of original names for each instruction. https://sannybuilder.com/dev/sa_opnames.txt As we did not have this information 20 years ago opcode syntax was invented where each command is represented as an opcode number followed by inline words to describe parameters. If we had command names from the start no one had to guess what each opcode means and invent confusing or misleading names create_thread etc.

So the keyword is a name for the particular script instruction. Some of the old keywords are obsolete and does not match original Rockstar naming convention (such as create_thread vs start_new_script). Keywords added recently for CLEO 4 and CLEO+ plugin as well as newOpcodes follow this convention and are as close to the original schema as possible. GTA SA SCR mode also uses original keywords now. We need to update other modes (GTA3, VC, SA) with the original keywords but in some cases it's not straightforward due to rearranged parameters.

If keywords are hard to memorize (which I totally agree, no one should remember each of them), classes are there to help. They group similar commands. In your example, for 0D01 we could make a class Matrix and add new method Rotate so you can write:

 Matrix.Rotate(0@, 1.0, 0.0, 0.0, 45.0)

I think this should be easier to use than opcode syntax or keywords syntax.

[x87]

Therefore, only the descriptive characters at the beginning can be used as keywords.

I get what you mean. But it contradicts with the opcode syntax we had for years. Considering these two examples:

09E2: $PARKED_IMPEXPM_CARS[0] = parked_car_generator_w_numberplate #EUROS colors -1 -1 force_spawn 0 alarm 50 door_lock 0 min_delay 0 max_delay 10000 plate "IMPEXP__" at 0.0 0.0 0.0 angle 180.0
09E2: CREATE_CAR_GENERATOR_WITH_PLATE 0.0 0.0 0.0 180.0 587 -1 -1 0 50 0 0 10000 "IMPEXP__" $car

I think having words before each parameter is much easier and helpful rather than have only a keyword in the beginning.

[XMDS]

If keywords are hard to memorize (which I totally agree, no one should remember each of them), classes are there to help. They group similar commands. In your example, for 0D01 we could make a class Matrix and add new method Rotate so you can write:

 Matrix.Rotate(0@, 1.0, 0.0, 0.0, 45.0)

I think this should be easier to use than opcode syntax or keywords syntax.

This means that classes.db should be updated and separate classes.db should be made for each extension to match the keywords.

If you want to add original keywords to all modes, I noticed that the SASCR mode has completely used keyword syntax. Will you update SCM.ini and OP.txt in the normal SA, VC, and GTA3 modes (just like SASCR mode), or will you still retain the current op syntax? If you choose not to keep it, it may be unaccustomed to some people. In fact, the keywords in the current normal mode are still meaningless.

[x87]

This means that classes.db should be updated and separate classes.db should be made for each extension to match the keywords.

this is exactly my long-term goal - to have the complete classes library that covers all opcodes in all modes (except *SCR).

Will you update SCM.ini and OP.txt in the normal SA, VC, and GTA3 modes (just like SASCR mode), or will you still retain the current op syntax?

I'm going to preserve existing syntax in GTA 3, VC and SA modes. If needed we could have new GTA 3 SCR and VC SCR modes with original naming.

[x87]

$OPCODE with file path acts as an alternative way of loading custom opcode definitions.

{$USE custom} - loads opcodes.ini and other syntax files from {edit_mode_dir}\custom directory {$O custom.ini} - loads custom.ini in the same directory as the script (only opcode definitions)

[Aldrin-John-Olaer-Manalansan]

We could also agree on some implicit way of adding extensions, for example, have a subfolder inside edit mode directory named extensions where you create a new folder, say, cleo+ and add new files here:

- Sanny Builder\
-- data\
--- sa\
---- extensions\
----- cleo+\
       |- opcodes.txt
       |- templates.txt
       |- keywords.txt
       |- opcodes.ini

Sanny would automatically register a new extension as if it was defined explicitly in the modes.xml

I'm looking forward on this implementation. It is very tiring to alter those 4 file definitions just to add your custom opcodes.

Also regarding to decompiling scripts. For example there are multiple extensions that have similar opcodes(Opcode Clash) and the script being decompiled has that opcode. My idea of making Sanny builder deal with it is:

1) Prompt the user which extensions should be used before decompiling the script.

Or what we can do:

Sample Case: extensions\thatextension\SASCM.ini - 0EFE=3,p1 %s p2 %s p3 %s extensions\myextension\SASCM.ini - 0EFE=3,p1 %d p2 %f p3 %d extensions\anotherextension\SASCM.ini - 0EFE=5,p1 %d p2 %s p3 %d p4 %f p5 %d

2) Count the number of parameters that the Opcode 0EFE of the script being decompiled. Then Read each extension's SASCM.ini's Opcode 0EFE number of parameters.

• If only one extension's opcode 0EFE has the same number of parameters the as the script being decompiled's Opcode 0EFE(eg. script being decompiled 0EFE=5,and anotherextension\SASCM.ini has 0EFE=5). Then extension's opcode definition will be used by Sanny Builder.
• If multiple extension's opcode 0EFE has the same number of parameters the as the script being decompiled's Opcode 0EFE(eg. 0EFE=3). Then this Sanny Builder will Warn the user what to do with it or Prompt to select which extension to use. If No Extension has Opcode 0EFE. set it to hex when decompiled

[XMDS]

@Aldrin-John-Olaer-Manalansan I think that new OPs made by third-party plug-ins are likely to be downward compatible. Even if several plug-ins have the same operation, they are also compatible. In addition, I think different opcode plugin writers should pay attention to the registered OP ID. As far as I know, for mainstream plugins, the new OP ID they bring is usually not repeated. Even now CLEO+ has expanded to 0EFF. We can also continue to develop towards 0FXX-1XXX.

[Aldrin-John-Olaer-Manalansan]

@Aldrin-John-Olaer-Manalansan I think that new OPs made by third-party plug-ins are likely to be downward compatible. Even if several plug-ins have the same operation, they are also compatible. In addition, I think different opcode plugin writers should pay attention to the registered OP ID. As far as I know, for mainstream plugins, the new OP ID they bring is usually not repeated. Even now CLEO+ has expanded to 0EFF. We can also continue to develop towards 0FXX-1XXX.

That is a good point for "active developers" but we also need to take note that there can be extensions that are already discontinued. One example is FYP's SAMPFUNCS which started on 2013 and was discontinued on 2017 with its final version 5.4.1 FINAL, it has a lot of opcodes.

Currently, SAMPFUNCS clashes on two opcodes, 0B20 and 0B21 which are defined on clipboard extension.

[x87]

@Aldrin-John-Olaer-Manalansan Hi 👋 Thank you for the feedback. Here is what I think.

Count the number of parameters that the Opcode 0EFE of the script being decompiled.

the issue here is that the compiled code does not store the number of parameters for the opcode. This is why we rely on opcode definitions (*SCM.ini). So in order to correctly read the opcode data and proceed to the next one, we need to know in advance how much to read. Here is the dilemma as the same opcode may have different number of parameters depending on what extension was used to compile it.

Prompting user to select an extension is potentially a good option but I suppose it will not be accepted as an user-friendly option. Adding new error for missing {$USE} directive in 3.6.0 caused a lot of confusion and adding a new prompt during disassembling could also be seen negatively.

[Aldrin-John-Olaer-Manalansan]

Prompting user to select an extension is potentially a good option but I suppose it will not be accepted as an user-friendly option. Adding new error for missing {$USE} directive in 3.6.0 caused a lot of confusion.

Yes, I also see the declaration of used extension from scripts to be a bit Redundant. An example case is: 1) Opcode 0FFF is Registered at /extension/my extension/SASCM.ini 2) Opcode 0FFF is unique among all extensions(meaning there are no same opcode clashing this opcode) 3) I used 0FFF on my script but I forgot to put {$USE my extension} 4) Sanny Builder should automatically detect that Opcode 0FFF belongs to myextension. By doing this, My users will not need to put that $USE header 5) I think {$USE extension} should only be needed if the script being compiled uses an opcode that exists within multiple extensions the Sanny Builder Knows?

and adding a new prompt during disassembling could also be seen negatively.

If it is one step for being organized then I don't think it has a negative impact. And besides, users who will decompile a certain script already know what extensions are required for that script to work(script authors always tell it's users what the script requires). And the require of prompting for which extension should be used for decompiling can be Minimized by:

• I am decompiling target.cs
• it has an unknown opcode 0EFE And 0FDD And 0FFF
• 0EFE is registered on three different extensions named extension1,extension2,extension3
• 0FDD is registered on two different extensions named extension3,extension4
• 0FFF is registered at only one extension named extension5 1) Sanny builder will only ask which among the four extensions(extension1|extension2|extension3|extension4) should be used. 2) Sanny builder will automatically use extension5

[x87]

I think {$USE extension} should only be needed if the script being compiled uses an opcode that exists within multiple extensions the Sanny Builder Knows?

the primary goal of adding extensions was not to deal with opcode conflicts, but rather make it explicit that a script uses non-standard commands. That is, does not matter if it's a main.scm or cleo script if you use opcodes that require an additional software to the vanilla game, you might want to be aware of it. Specifically it was caused by adding hundreds of new opcodes from CLEO+ where novice scripters could wrongly assume they use standard CLEO library opcodes but in reality they originate from the plugin. CLEO opcodes would require {$USE} too but I figured {$CLEO} achieves essentially the same goal so the latter implicitly adds {$USE CLEO} otherwise you still need it in a main.scm for example.

Declaring custom extensions/packages is common to any programming language out there, in a form of import/include statements.

script authors always tell it's users what the script requires

exactly, they now gain this knowledge even if they did not realize it before.

I would think of automating $USE directives when you add a new command from Opcode Search or F1 search, to make it less annoying but I would not want to make all extensions implicitly available as it was before. With command definitions files split in different directories IDE will not suggest extension opcode until you explictly import them into your script (with the $USE).

[x87]

Sanny builder will only ask which among the four extensions(extension1|extension2|extension3|extension4) should be used.

I think it's a good idea. So we could also leverage the information about opcodes we've seen before and if they already come from one of the extensions then we could automatically pick that. E.g.:

extension A: 0001 0002

extension B 0001 0002 0003

script 1: 0003, 0001, 0002

script 2: 0001, 0002, 0003

For the script 1 we find 0003 first so we pick extension B as the choice for this script. And we continue to do so with 0001 and 0002 without prompting a user. But for the script 2 we can't choose the extension so we ask the user to pick one.

[Aldrin-John-Olaer-Manalansan]

Declaring custom extensions/packages is common to any programming language out there, in a form of import/include statements.

You are right about that, scripters just need to adapt on what's new. I don't actually find this hard and confusing, and so as other scripters will get used to it later on...

Oh, also, I don't know if this is the right thread to ask for requests. But are you planning on adding FYP's SAMPFUNCS extension to your defined sets of extension?

[x87]

But are you planning on adding FYP's SAMPFUNCS extension to your defined sets of extension?

I don't have such plans. The goal is to make the process of adding new extensions as easy as just copying a folder into Sanny Builder data directory. If anyone is interested to convert SAMPFUNCS into a proposed format they could do so and distribute as a prepared package.

I don't want to bear a burden of maintaining all known opcode plugins.

[Aldrin-John-Olaer-Manalansan]

If anyone is interested to convert SAMPFUNCS into a proposed format they could do so and distribute as a prepared package.

I can create all definitions of SAMPFUNCS Extension for the upcoming Sanny Builder version.

For the mean time, I will wait for the final proposed format.

[x87]

@Aldrin-John-Olaer-Manalansan Ok. Can you please open a new ticket for this request similar to #71?

[Aldrin-John-Olaer-Manalansan]

@Aldrin-John-Olaer-Manalansan Ok. Can you please open a new ticket for this request similar to #71?

Requested ticket #112

[x87]

Moving this ticket out of v3.7 as I need more time to come up with a solution to some cases.

• [ ] handle opcode clashes during disassembling
• [ ] split results of make_opcodes https://github.com/sannybuilder/make_opcodes/issues/3
• [ ] make Language service to collect information on used extensions and update autocomplete (opcodes list, keywords, classes)
• [ ] have an option in Opcode Search tool to search across all extensions/only currently used ones
• [ ] provide a compiler prompt to add a particular extension to the file when it finds an unknown opcode

[x87]

@OrionSR

Are Restored Opcodes currently supported by Sanny?

they are now as of https://github.com/sannybuilder/dev/releases/tag/v3.7.0-alpha.3

[x87]

I'd like to push this idea forward and start off of a conservative approach. Given a number of challenges we could limit the proposal to the following:

1. allow multiple <opcodes> definitions.

   <opcodes>@sb:\data\vc\opcodes.ini</opcodes>
   <opcodes>@sb:\data\vc\plugins\*.opcodes.ini</opcodes> // not sure about wildcard support

   • Each file overwrites previous definitions in case of id clashes. To resolve the clash differently, reorder the <opcodes>.

2. allow self-declarative mode folders. Instead of defining each mode in modes.xml, a user could create a new folder under SB\data, place a file called mode.xml (same format as <mode> in modes.xml), and select it from SB's list of modes.

• Each mode could still extend an existing mode with <extends>. In addition to the mode id, an argument could be a path to mode.xml of the mode to extend (i.e. extends="../sa/mode.xml")
• Users can delete modes they are not interested in (another solution to #184), either by deleting the folder or renaming mode.xml file
• modes.xml won't be present in the future versions

[OrionSR]

Users can delete modes they are not interested in (another solution to https://github.com/sannybuilder/dev/issues/184), either by deleting the folder or renaming mode.xml file.

My thought was to add a data\disabled folder for edit modes I don't intend to use regularly.

A user could quickly get into trouble by disabling modes that are extended. And renaming mode.xml seems likely to cause similar issues.

Still, I like the basic idea of making custom edit modes easier to implement. I've been putting off customizing my edit modes since it tends to lead to conflicts when sharing scripts. Perhaps it's time to revisit the idea to provide examples for the new feature.

Got any thoughts on implementing original variable names for SA?

[x87]

Got any thoughts on implementing original variable names for SA?

you mean #195? Haven't looked closely yet, sorry.

[x87]

closing as the limited proposal has been implemented in 3.9.0