BattletechModders / ModTek

Mod system for HBS's PC game BattleTech.
GNU Lesser General Public License v2.1
121 stars 34 forks source link

Update Documentation to use Traverse instead of "ReflectionHelper" #56

Closed mpstark closed 6 years ago

mpstark commented 6 years ago

Traverse is better then using reflection.

CptMoore commented 6 years ago

traverse is not necessary anymore to access private fields

using three "_" underscores before a parameter name in a signature of a postfix/prefix will search for that name as a private field on the class.

(not sure if thats already in the doc or not)

gnivler commented 6 years ago

Sounds like I might be able to do this, just not sure where to go to update docs. Just fork and PR? Thanks

mpstark commented 6 years ago

The wiki isn't really PR'able I don't think. You could copy paste the sections into this ticket and then I could just copy paste them onto wiki.

gnivler commented 6 years ago

I hope this isn't a total disaster of a paste job. I cloned the wiki to GitHub desktop but it's not a fork, I can't just push clearly, and am thinking getting more involve has a low return. So here's some plaintext that I believe will go straight into the wiki... edit: plaintext is getting markeddowned for better or worse if you want a pastebin, np

gnivler commented 6 years ago

Installing BattleTech Mod Loader

BTML releases are hosted on GitHub here. Make sure that you are downloading the actual release (will be something like "BattleTechModLoader-v0.2.0.zip") instead of the source. Check the zip file, it should contain an .exe and a couple .dlls.

Find where BattleTech has been installed to. You're looking for a folder called BATTLETECH. Default paths are:

In this directory, there will be a directory: BATTLETECH\BattleTech_Data\Managed\. It will have lots of .dll files in it, including the one that we'll be injecting into: Assembly-CSharp.dll.

Drag all of the files from the downloaded BTML zip into this folder. There should be 0Harmony.dll, BattleTechModLoader.dll and BattleTechModLoaderInjector.exe. They are all necessary.

Run BattleTechModLoaderInjector.exe after it has been moved into the directory BATTLETECH\BattleTech_Data\Managed\. It'll pop open a console window and inject into Assembly-CSharp.dll after backing it up. The file may be a little smaller after injecting, that's normal. You can delete the injected file and rename the .orig file back, or run BattleTechModLoaderInjector.exe /restore from a command prompt (Win+R and type cmd ). Deleting the .orig file will invalidate the /restore feature.

Whatever you do, do not ever put BattleTechModLoader.dll or 0Harmony.dll into the BATTLETECH\Mods folder. This will create an infinite loop of BTML loading itself -- it's not pretty.

BTML is now installed.

ModTek

Download ModTek.dll from the GitHub Releases page here. This is not in a zip folder or compressed, it's just a .dll file.

Navigate back to BATTLETECH\. If there isn't a BATTLETECH\Mods\ folder, create it. Move ModTek.dll into the BATTLETECH\Mods\ folder.

ModTek is now installed. If BTML is also installed and injected, ModTek will add a "/W MODTEK" with the ModTek version number to the games main menu version number in the bottom left. If you don't see ModTek list in the bottom right corner, BTML is likely not injected or you put ModTek.dll in the wrong place. Double-check the previous instructions.

Installing a ModTek Mod

Download the ModTek mod that you want to install. All ModTek mods exist in their own subfolders inside BATTLETECH\Mods. If you unzip the mod, ensure that it goes into its own folder.

The resulting mod-folder must have a mod.json file in the first level -- something like this: BATTLETECH\Mods\ModThatYouJustDownloaded\mod.json. If it does not, then something is wrong, maybe your mod-folder contains the actual mod folder. If this is the case, just move the contents down a level.

Your ModTek mod is now installed.

gnivler commented 6 years ago

This will be a guide for writing a fairly simple mod -- making the Leopard have a 300 ton drop-limit while playing the campaign/simgame.

Getting Started

The easiest way to change how the game works is to see if you can do it by editing the game's JSON and writing a ModTek JSON mod -- some examples are here. You should only spend your time and effort if that time and effort is required to get your project working. HBS has provided a lot of tweaks to the way that the game works in JSON files and it's not uncommon that they've got something already in place. In this case however, there are none used by the game so we'll just have to make it.

Tools

This tutorial/walkthrough uses:

Do Your Research

In order to modify the way that something works, you should probably learn how it works to begin with. Use dnSpy's search and analyze functionalities to get familiar with the code. Note the pulldowns in the search feature, where you can filter for certain members, or numbers and strings. A more detailed primer for navigating BattleTech's code with dnSpy will probably come.

In our case, we want to simply add the condition that you cannot launch a mission while overweight, which pretty closely matches the code for not being able to launch without a Mechwarrior in a 'Mech. By digging around, I found BattleTech.UI.LanceConfiguratorPanel.ValidateLance(), which is the function that is called to see if the configured Lance is valid or not.

Setting Your Project Up

Create a new project and solution with Visual Studio. You should target .net 3.5, as this is what the game is using.

On the right, make sure that you add references to the 0Harmony.dll and Assembly-CSharp.dll. If you will be using the settings from your mod.json, you will also need Newtonsoft.Json.dll. All of these files can be found in your BATTLETECH\BattleTech_Data\managed directory (except Harmony if you haven't installed BTML, which you should have).

It's a hassle to change the other references like System and such to match the installed game and unless you're doing something special, you can skip changing the references. You do not need to reference BTML or ModTek.

Actually Writing Your Mod

Now that you understand how the functionality works, you can change it. In our case, we want ValidateLance to return false when the lance is overweight, as well as fill in the same values that the function already does (i.e. we want to have exactly the same side effects as if the code was written in the method itself). It would be relatively easy to just hop into changing it in dnSpy and recompiling the method -- but we can't do that in this case, since we want to have a seperate .dll that does it at runtime.

That's why we're using Harmony -- it allows you to "hook" onto methods before and after they are called, as well as to directly modify the executed IL code with a transpiler. The 'hooks' before are called Prefixes; they can modify the parameters passed into the function, as well as actually prevent the original code from being called, and the hooks after are called Postfixes; which can modify what the function returns. You can learn more about Harmony from looking at it's wiki and looking through other people's Harmony-based mods.

Since ValidateLance doesn't have parameters, we still want the original code to execute, and we want to change ValidateLance's return value, we'll use a postfix patch, which something like this:

[HarmonyPatch(typeof(LanceConfiguratorPanel), "ValidateLance")]
public static class LanceConfiguratorPanel_ValidateLance_Patch
{
    public static void Postfix(LanceConfiguratorPanel __instance, ref bool __result)
    {
    }
}

Right now, it doesn't do anything and it won't even get called. You'll notice that it's static class with a Postfix method, that it's got an annotation that has the type (we're using BattleTech.UI; at the top of the file) and method name passed as a (magic) string. This is to tell Harmony which method specifically that we want to patch (if it is overloaded, you'll need to provide an array of parameter types too!). But first, in order for this patch to even get setup, we'll need to setup Harmony to read these annotations.

BTML/ModTek will call Init(void), Init(string, string) or you can setup a custom entry point in your mod.json file. We'll just use the default entry point with the two string parameters, they'll be useful to us later. So we'll setup a 'main' static class that contains an Init(string, string).

public static class DropLimit
{
    public static void Init(string directory, string settingsJSON)
    {
        var harmony = HarmonyInstance.Create("io.github.mpstark.DropLimit");
        harmony.PatchAll(Assembly.GetExecutingAssembly());
    }
}

This will instantiate a HarmonyInstance with your unique identifier (Harmony recommends the reverse domain notation, but any unique string will work), as well as search your entire assembly for classes with annotations like the one we setup on our patch class. Now, if we compiled our code and dragged our .dll into our mod folder with our mod.json, our blank method would be called every time after ValidateLance is called.

Let's make our patch do something. First, take a look at the parameters that I setup. Both are special parameters given by Harmony -- the first is to get the object that this particular call is for, sort of a this for Harmony. The second is a ref bool type, because the function returns a bool value and we want to be able to change it.

Returning to dnSpy, we need to figure out exactly what we need to do to emulate what the method does when it detects an error -- since it doesn't just return false when it detects on error, it has other side-effects. Namely, it sets lanceErrorText to the error, and it also passes it to the headerWidget object, along with some other infomation. In order to do this correctly, we have to do all three things.

Once we start implementing this, we run immediately into the issue of accessing non-public fields. Because we have merely have a reference to __instance, we can only do the normal public things, and what we want to do is.. private.

Reflection to the rescue!

var field = __instance.GetType().GetField("fieldName", BindingFlags.NonPublic | BindingFlags.Instance);
var fieldValue = field.GetValue(__instance);
field.SetValue(__instance, value);

Since we just need to get and set some private variables, this is all we need to do. My friend /u/Morphyum from Reddit and the Reddit BattleTech Discord has written a small static helper class called "ReflectionHelper". We'll use it in our example instead of doing the reflection stuff a couple different times, since it gets old.

// CODE FROM MORPHYUM
public static class ReflectionHelper
{
    public static void SetPrivateField(object instance, string fieldname, object value)
    {
        var type = instance.GetType();
        var field = type.GetField(fieldname, BindingFlags.NonPublic | BindingFlags.Instance);
        field.SetValue(instance, value);
    }

    public static object GetPrivateField(object instance, string fieldname)
    {
        var type = instance.GetType();
        var field = type.GetField(fieldname, BindingFlags.NonPublic | BindingFlags.Instance);
        return field.GetValue(instance);
    }
}

He's also got stuff for private properties and private methods, but these are all that we need. Our new method looks something like this:

public static void Postfix(LanceConfiguratorPanel __instance, ref bool __result, ___loadoutSlots, ___headerWidget, ___lanceValid, ___lanceErrorText)  
// using triple underscores for parameters now automatically accesses private fields using Harmony.  _myFieldName would be accessed with 4 underscores.
{
    float lanceTonnage = 0;

    var mechs = new List<MechDef>();
    for (var i = 0; i < __instance.maxUnits; i++)
    {
        var lanceLoadoutSlot = ___loadoutSlots[i];

        if (lanceLoadoutSlot.SelectedMech == null) continue;

        mechs.Add(lanceLoadoutSlot.SelectedMech.MechDef);
        lanceTonnage += lanceLoadoutSlot.SelectedMech.MechDef.Chassis.Tonnage;
    }

    if (lanceTonnage <= 300) return;

    __instance.lanceValid = false;

    (___headerWidget as LanceHeaderWidget).RefreshLanceInfo(__instance.lanceValid, "Lance cannot exceed tonnage limit", mechs);

    ___lanceErrorText = "Lance cannot exceed tonnage limit\n";

    __result = __instance.lanceValid;
}

You'll notice that we had to do some extra stuff to satisfy the side effects of the original method, namely make a list of MechDefs to pass to RefreshLanceInfo. Compile, drag the compiled result to our mod folder, run the game and it works!

Making it better

Remember when I said that ModTek could pass you the settings json from the mod.json file? Let's use it! The easiest way is to create a settings class with some default values. We don't have to setup a constructor because one is generated for us for such a simple class.

internal class ModSettings
{
    public float MaxTonnage = 300;
    public bool OnlyInSimGame = true;
}
internal static ModSettings Settings = new ModSettings();
public static void Init(string directory, string settingsJSON)
{
    var harmony = HarmonyInstance.Create("io.github.mpstark.DropLimit");
    harmony.PatchAll(Assembly.GetExecutingAssembly());

    // read settings
    try
    {
        Settings = JsonConvert.DeserializeObject<ModSettings>(settingsJSON);
    }
    catch (Exception)
    {
        Settings = new ModSettings();
    }
}

This will use Newtonsoft.Json to create a new settings object for our mod, which will be stored in a our static class that holds Init. If the settings json has problems or doesn't exist, then we'll just use the default settings. Note that this sort of error handling is fast and loose and you shouldn't actually do it.

Now it's easy to change our patch to use these settings.

if (DropLimit.Settings.OnlyInSimGame && !__instance.IsSimGame)
    return;
// ...
if (lanceTonnage <= DropLimit.Settings.MaxTonnage)
    return;
gnivler commented 6 years ago

Before you read any of this, you should know that if HBS' base JSON is invalid (excepting for their comments that they pull out during runtime, and trailing commas, which are ignored), ModTek cannot patch it. ModTek tries to automatically fix missing commas.

ModTek makes mods that change JSON values or add new JSON files far, far better for the end-user. Instead of manually editing existing game files or replacing them with the ones that you provide, they can simply drag and drop your mod folder into the \BATTLETECH\Mods\ directory and ModTek will take care of the rest, both by adding new JSON files into the manifest and by merging JSON that matches existing files onto the game JSON. Uninstalling your mod or updating it is as simple as deleting the existing folder and/or replacing it.

Developing a JSON mod for ModTek is fairly straightforward and in this article/guide, we'll build three different mods. First, we'll make a simple mod that changes the StreamingAssets\data\constants\AudioConstants.json file as suggested by Alec Meer at RockPaperShotgun based on community findings. After that, we'll build a mod that changes a hardpoint on a mech chassis. Finally we'll add new variants of an existing chassis.

For all of the examples, we will first need to write a mod.json file for ModTek to read. Much more in-depth documentation for that is here.

Note: The log file at the path BATTLETECH\Mods\ModTek.log is very helpful for figuring out if something has worked or not. There is also the game log located at BATTLETECH\BattleTech_Data\output_log.txt that can tell you if something broke in the game.

EliminateAudioDelays, making simple changes to an existing JSON file

As described in the mod.json article, we'll setup ours to look like this:

{
    "Name": "EliminateAudioDelays",
    "Enabled": true,
    "Version": "0.1.0",
    "Description": "Get rid of those delays!",
    "Author": "mpstark",
    "Website": "www.github.com/mpstark/ModTek",
    "Contact": "fakeemail@fakeemail.com",
}

Since the file that we want to modify already exists in the game at the path BattleTech_Data\StreamingAssets\data\constants\AudioConstants.json, we'll simply use ModTek's implicit StreamingAssets mirror folder. Everything in this mirror of the existing files will be loaded without specifying it in the mod.json manifest, if it's in the same place with the same name. Additionally, you should not use the StreamingAssets folder for new files, just ones like AudioConstants.json that already exist.

So we'll just copy the existing file over to our mod folder in the path BATTLETECH\Mods\EliminateAudioDelays\StreamingAssets\data\constants\AudioConstants.json. Then we'll open that file with our favorite text editor (I love Visual Studio Code with the JSON linter!).

The AudioConstants.json file is pretty big and contains a lot of stuff -- I find that the collapsing blocks in my editor help close out things from the file that I don't care about. We're looking for five values, AttackPreFireDuration, AttackAfterFireDelay, AttackAfterFireDuration, AttackAfterCompletionDuration and audioFadeDuration which control the things that we want to change. Since we only want to change these five things and they're all in the first level (i.e. they're not nested in anything), we can simply delete everything except for those entries and edit them to have our values. Our resulting "partial" JSON file looks something like this:

{
   "AttackPreFireDuration" : 0,
   "AttackAfterFireDelay" : 0,
   "AttackAfterFireDuration" : 0,
   "AttackAfterCompletionDuration" : 0,
   "audioFadeDuration" : 0
}

Save the file and your mod is now complete. What happens when BattleTech loads the JSON is that ModTek intercepts it and "merges" your values onto what was loaded in mod load order. So the game initially sees the original file, then it'll merge the first loaded mod that affects that file, then the second, then the third, etc. As you can probably intuit, the last mod that changes a value is the value that is chosen, because it wrote over any other mod's value. There are some details about this process for JArrays (the ones that use the [] braces and don't have keys) that we'll talk about in the next example.

Also, before you distribute anything, you should probably check it over. VSCode has a linter extension for JSON, but if you have a different editor, you should give it a quick once-over in a linter like this free online one. A linter will try to point out things that are wrong, like you missed a comma somewhere or have a missing brace.

Changing how many 'Mech pieces it takes to salvage

If we want to change something that is in a second level, we have to include the parent object as well. In \simGameConstants\SimGameConstants.json, there is a single value, "DefaultMechPartMax", that changes how many pieces of salvage that it takes to combine and fix up a 'Mech, however, this is part of a larger object, called "Story". If we simply wanted to change this value and nothing else, our partial JSON would look like this:

{
    "Story" : {
        "DefaultMechPartMax" : 5
    }
}

Note that even though we specified the "Story" object, we didn't have to include all of the values of it.

Modding the AS7-D head hardpoint, or, why HBS should have made Locations a JObject

Based on the previous examples, you would expect everything to follow the same basic pattern, remove everything that you're not changing, and change the things that you are. And for anything that is a JObject, i.e. the things surrounded by {} and that have "Keys" on the left, a colon : in the middle, and "Values" on the right.

Unfortunately, with JArrays, which are surrounded by [] braces, things are a little bit more complicated then they could have been. Since they don't have keys, they can't be indexed well, and since their order is arbitrary and could change between game version, they can't be relied upon to be in the same spots. With some simple arrays, you can make assumptions on how a JSON merge should go, but because of the relatively complex nature of BattleTech's JSON, any change that you make to an array, replaces that array with yours, including things you didn't edit.

Our goal is to change the AS7-D's head to have a hardpoint for support called "anti-personal" in the files. In order to do this though, we have to specify the entire locations array, which means that only one mod can change these each arrays at a time.

Here's our modded chassis file for the AS-7D which we would put into our mod folder at MyModFolder\StreamingAssets\data\chassis\chassisdef_atlas_AS7-D.json. You'll still obviously need to provide a mod.json file as well.

{
    "Locations": [
        {
            "Location": "Head",
            "Hardpoints": [
                {
                    "WeaponMount": "AntiPersonnel",
                    "Omni": false
                }
            ],
            "Tonnage": 0,
            "InventorySlots": 1,
            "MaxArmor": 45,
            "MaxRearArmor": -1,
            "InternalStructure": 16
        },
        {
            "Location": "LeftArm",
            "Hardpoints": [
                {
                    "WeaponMount": "Energy",
                    "Omni": false
                },
                {
                    "WeaponMount": "AntiPersonnel",
                    "Omni": false
                }
            ],
            "Tonnage": 0,
            "InventorySlots": 8,
            "MaxArmor": 170,
            "MaxRearArmor": -1,
            "InternalStructure": 85
        },
        {
            "Location": "LeftTorso",
            "Hardpoints": [
                {
                    "WeaponMount": "Missile",
                    "Omni": false
                },
                {
                    "WeaponMount": "Missile",
                    "Omni": false
                }
            ],
            "Tonnage": 0,
            "InventorySlots": 10,
            "MaxArmor": 210,
            "MaxRearArmor": 105,
            "InternalStructure": 105
        },
        {
            "Location": "CenterTorso",
            "Hardpoints": [
                {
                    "WeaponMount": "Energy",
                    "Omni": false
                },
                {
                    "WeaponMount": "Energy",
                    "Omni": false
                }
            ],
            "Tonnage": 0,
            "InventorySlots": 4,
            "MaxArmor": 320,
            "MaxRearArmor": 160,
            "InternalStructure": 160
        },
        {
            "Location": "RightTorso",
            "Hardpoints": [
                {
                    "WeaponMount": "Ballistic",
                    "Omni": false
                },
                {
                    "WeaponMount": "Ballistic",
                    "Omni": false
                }
            ],
            "Tonnage": 0,
            "InventorySlots": 10,
            "MaxArmor": 210,
            "MaxRearArmor": 105,
            "InternalStructure": 105
        },
        {
            "Location": "RightArm",
            "Hardpoints": [
                {
                    "WeaponMount": "Energy",
                    "Omni": false
                },
                {
                    "WeaponMount": "AntiPersonnel",
                    "Omni": false
                }
            ],
            "Tonnage": 0,
            "InventorySlots": 8,
            "MaxArmor": 170,
            "MaxRearArmor": -1,
            "InternalStructure": 85
        },
        {
            "Location": "LeftLeg",
            "Hardpoints": [],
            "Tonnage": 0,
            "InventorySlots": 4,
            "MaxArmor": 210,
            "MaxRearArmor": -1,
            "InternalStructure": 105
        },
        {
            "Location": "RightLeg",
            "Hardpoints": [],
            "Tonnage": 0,
            "InventorySlots": 4,
            "MaxArmor": 210,
            "MaxRearArmor": -1,
            "InternalStructure": 105
        }
    ]
}

Even though we didn't change anything other than add that single hardpoint to the head, we still had to restate the entire array. Which kind of sucks.

Adding new 'Mech loadouts

As always, each individual mod that you ship out as a mod folder needs a mod.json file. In the previous cases, we didn't need to specify a manifest because we were changing existing game files. If we want to add new files, then we need to add a manifest to the mod.json file.

Say that we wanted to add some 'Mech loadouts for existing chassis. After writing the Mechdef files, with standard BattleTech modding, you'd have to add each to the VersionManifest.csv or add an addendum. You can think of the mod.json manifest as exactly the same thing, except you don't have to use .csv, you don't have to fake a time and date, you don't have to count commas, and most importantly you can specify entire folders for a single type.

Note: The StreamingAssets folder should not contain new entries, just ones from the base game that you want to modify, even if you point at them in the manifest.

{
    "Name": "TheGreatestMech",
    "Enabled": true,
    "Manifest": [
        { "Type": "MechDef", "Path": "MyMechs\\" }
    ]
}

This will now add all of the .json files in the MyModFolder\MyMechs to the VersionManifest at runtime automatically. Pretty sweet huh?

A corollary to this

Using the same mechanism, you can add to a specific manifest addendum or create a manifest addendum by using the AddToAddendum field in each manifest entry. This is particularly useful for adding emblems to the game, as "deim0s" from the BattleTech Reddit's Discord channel discovered. Since emblems need to be added twice here, once as a type "Sprite" and once as a type "Texture2D".

{
    "Name": "MyEmblems",
    "Enabled": true,
    "Manifest": [
        { "Type": "Texture2D",  "Path": "emblems\\", "AddToAddendum": "PlayerEmblems" },
        { "Type": "Sprite",     "Path": "emblems\\", "AddToAddendum": "PlayerEmblems" }
    ]
}
gnivler commented 6 years ago

A couple dirty diffs to give you an idea where I was

image image image (and more in that one)

image

gnivler commented 6 years ago

and at a file level in full ModTek.wiki.zip

I didn't know what to do with the stuff about ReflectionHelper and credit to Morph so left it and just updated the code section

gnivler commented 6 years ago

Better I hope - this one is compiling


public static void Postfix(LanceConfiguratorPanel __instance, ref bool __result, LanceLoadoutSlot[] ___loadoutSlots, LanceHeaderWidget ___headerWidget, string ___lanceErrorText)  
// using triple underscores for parameters now automatically accesses private fields using Harmony.  _myFieldName would be accessed with 4 underscores.
{
    float lanceTonnage = 0;

    var mechs = new List<MechDef>();
    for (var i = 0; i < __instance.maxUnits; i++)
    {
        var lanceLoadoutSlot = ___loadoutSlots[i];

        if (lanceLoadoutSlot.SelectedMech == null) continue;

        mechs.Add(lanceLoadoutSlot.SelectedMech.MechDef);
        lanceTonnage += lanceLoadoutSlot.SelectedMech.MechDef.Chassis.Tonnage;
    }

    if (lanceTonnage <= 300) return;

    __instance.lanceValid = false;

    ___headerWidget.RefreshLanceInfo(__instance.lanceValid, "Lance cannot exceed tonnage limit", mechs);

    ___lanceErrorText = "Lance cannot exceed tonnage limit\n";

    __result = __instance.lanceValid;
}
mpstark commented 6 years ago

I've updated the documentation. Thanks!