Selection Limit higher than 100

[PodeCaradox]

Hello LitzeYT,

is it maybe possible to set the Selection Limit of Soldiers higher than 100? Maybe as a Option in the Patcher.

Also what are you using for your UI? Maybe i can help you out with a WPF GUI and a simpler interface that let you make the changes via byte changes on the Actual Exe(it will first create a Save exe offcourse) not via copy/replace it. The GUI than also can support multi Languages. Its like the Unofficial Patch(https://github.com/Sh0wdown/UnofficialCrusaderPatch) for Stronghold if you interested ;)

Thanks for your work and have a nice Day :)

[nyfrk]

is it maybe possible to set the Selection Limit of Soldiers higher than 100? Maybe as a Option in the Patcher.

Unfortunaly this is pretty much hardcoded in the game and some more work is required. The limit is coded at Opcode S4_Main.exe+0xED23E in the line cmp eax,100 (AOB Sig: 83 F8 64 0F 83 84 00 00 00). We can simply change it as all the selected elements are stored in a vector that simply expands when more elements become selected. However increasing it leads to a crash (fatal error) at S4_Main.exe+0x25F480 (AOB Sig: FF 50 04 89 85 43 85 00 00 85 C0 0F 84) when executing call [eax+0x04]. Thats because it comes to a buffer overflow at Opcode S4_Main.exe+0x26045B (AOB Sig: 89 B0 40 6D A7 01 8A 43 15 84 C0) by writing at [S4_Main.exe+0x1057524] which corrupts the pointer (esi) loaded by Opcode S4_Main.exe+0x25F450 (AOB Sig: 8B 35 24 75 A7 01 85 F6 0F 84 A4 01 00 00). We therefore need to fix the buffer overflow either by stopping to draw the health bubbles for selections greater than 100 or by some other means. Maybe we could find all the hardcoded places where the static buffer is accessed. We could than map it to some larger memory section or even to a vector. However im not really that much motivated in doing that work.

Note, when changing the limit do not forget to also change the number of units that can be processed per mouse click. Its configured by the DWORD at S4_Main.exe+0xEC8D0. Otherwise you can select more than 100 units but only the first 100 units will move when doing a rightclick somewhere.

Hope it helps you in creating a patch for it.

MD5 of my S4_Main.exe: bcdd32fd35f85a8eb96641dcc4eacfa9

[LitzeYT]

is it maybe possible to set the Selection Limit of Soldiers higher than 100? Maybe as a Option in the Patcher.

Unfortunaly this is pretty much hardcoded in the game and some more work is required. The limit is coded at Opcode S4_Main.exe+0xED23E in the line cmp eax,100 (AOB Sig: 83 F8 64 0F 83 84 00 00 00). We can simply change it as all the selected elements are stored in a vector that simply expands when more elements become selected. However increasing it leads to a crash (fatal error) at S4_Main.exe+0x25F480 (AOB Sig: FF 50 04 89 85 43 85 00 00 85 C0 0F 84) when executing call [eax+0x04]. Thats because it comes to a buffer overflow at Opcode S4_Main.exe+0x26045B (AOB Sig: 89 B0 40 6D A7 01 8A 43 15 84 C0) by writing at [S4_Main.exe+0x1057524] which corrupts the pointer (esi) loaded by Opcode S4_Main.exe+0x25F450 (AOB Sig: 8B 35 24 75 A7 01 85 F6 0F 84 A4 01 00 00). We therefore need to fix the buffer overflow either by stopping to draw the health bubbles for selections greater than 100 or by some other means. Maybe we could find all the hardcoded places where the static buffer is accessed. We could than map it to some larger memory section or even to a vector. However im not really that much motivated in doing that work.

Note, when changing the limit do not forget to also change the number of units that can be processed per mouse click. Its configured by the DWORD at S4_Main.exe+0xEC8D0. Otherwise you can select more than 100 units but only the first 100 units will move when doing a rightclick somewhere.

Hope it helps you in creating a patch for it.

MD5 of my S4_Main.exe: bcdd32fd35f85a8eb96641dcc4eacfa9

Thank you very much for your detailed answer. You have arrived at the exact same problem we are stuck with. It just happened that the game crashes when 102 units are reached.

Maybe one solution would be to compare two versions. Between 1.6 and 1.8 of the original version of Settlers IV the limit was raised/lowered, unfortunately the lore doesn't quite agree and I can't get the old version to run. So you could compare the ExE, the Sigs are the same in the Uplay version, but the Sigs are shifted.

Do you own Discord, then maybe we could talk about a solution together ?

Best regards LitZe

[nyfrk]

I do not see how comparing with another binary can benefit us. We already identified the problem and to properly solve it we must add padding to extend the static buffer. However to do this, we would have to shift all memory behind the buffer a little further. Without source code this is almost impossible.

My suggestion therefore was that we add our very own "unlimited vector" while keeping the original one with the 100 units limit in place. Whenever we do right click, instead of using the original vector, we use our vector. Therefore more than 100 units can move. This approach has a few advantages:

• Reasonable amount of work
• No unexpected crashes, as we just mod those pieces, that we really want to influence
• We could add custom filters for our vector like #3 suggests (or we could allow thieves and soldiers in the very same selection etc)
• Other stuff that is based on static 100-unit buffers (like group selections) would still work, (they would just only work for the first 100 units) thus preventing unexpected overflows. However we could decide to add those features later too by using our own buffers!

We would have to mod only those places to use our vector:

• Movement Command (when doing right click)
• Health bubble drawing
• Unit Summary UI (that one where the amount of units is listed)

I cannot find the source code for your project so i cannot add a pull request. But i can show you a demo how this could work. Note that for this proof of concept demo I only added our unlimited vector to the box selection tool and the movement command assignment. The unit summary ui / health bubbles are not implemented yet. Proof of concept: https://youtu.be/C6vbXYQ-9lA

The addresses and offsets i modified are already described in my previous post.

Note that there would be more work to do:

• Not every selection is done by a box select. Other selections (eg left clicking a single unit) must properly manipulate our unlimited vector too
• Selections are modified by the game eg when entering ships or when units die. This must properly manipulate our custom vector too
• Ships and war machines must be treated with care in selections. In my demo I did not consider them which may make them glitchy. However i did a few quick tests with war machines and they seem to still work.

[LitzeYT]

I do not see how comparing with another binary can benefit us. We already identified the problem and to properly solve it we must add padding to extend the static buffer. However to do this, we would have to shift all memory behind the buffer a little further. Without source code this is almost impossible.

My suggestion therefore was that we add our very own "unlimited vector" while keeping the original one with the 100 units limit in place. Whenever we do right click, instead of using the original vector, we use our vector. Therefore more than 100 units can move. This approach has a few advantages:

• Reasonable amount of work
• No unexpected crashes, as we just mod those pieces, that we really want to influence
• We could add custom filters for our vector like #3 suggests (or we could allow thieves and soldiers in the very same selection etc)
• Other stuff that is based on static 100-unit buffers (like group selections) would still work, (they would just only work for the first 100 units) thus preventing unexpected overflows. However we could decide to add those features later too by using our own buffers!

We would have to mod only those places to use our vector:

• Movement Command (when doing right click)
• Health bubble drawing
• Unit Summary UI (that one where the amount of units is listed)

I cannot find the source code for your project so i cannot add a pull request. But i can show you a demo how this could work. Note that for this proof of concept demo I only added our unlimited vector to the box selection tool and the movement command assignment. The unit summary ui / health bubbles are not implemented yet. Proof of concept: https://youtu.be/C6vbXYQ-9lA

The addresses and offsets i modified are already described in my previous post.

Note that there would be more work to do:

• Not every selection is done by a box select. Other selections (eg left clicking a single unit) must properly manipulate our unlimited vector too
• Selections are modified by the game eg when entering ships or when units die. This must properly manipulate our custom vector too
• Ships and war machines must be treated with care in selections. In my demo I did not consider them which may make them glitchy. However i did a few quick tests with war machines and they seem to still work.

https://github.com/LitzeYT/Settlers4Patch/tree/SourceCode

I just created another branch, I am a bit inexperienced with github, I hope you can start a pull request, there are now all the single files that have been patched.

You are right with your comments, it is better to do it this way.

Is it possible that you provide me with your patched version so I can have a closer look at it ? Then I could look at the other problems that are still to be done, like the ones you mentioned.

Strangely enough, I have already managed that it is possible to select more than 100 units of the same type with Alt+left click. Unfortunately, the problem with me was always the box selection.

I think it shouldn't be the biggest problem if you can still only choose 100 ships or 100 war machines, they are not played that often.

Unless you patch the ships to more than 15 occupants, I'm working on that right now, only the pointer is missing, then that's solved.

Best Regards LitZe

[LitzeYT]

Sorry for Close and reopen, i missclicked x)

[PodeCaradox]

@LitzeYT how exactly you apply your changes, do you do it by IDA or Hexeditor? my second suggestion was a application that do the changes with byte changes on the orginal exe without everytime use a new one. So you can set memory adresses and the changes from there like in the Crusader Modding Comunnity the UCP.

new BinHook(6)
{
    CMP(EAX, 5), //  cmp eax,05
    PUSH(FLAGS), //  pushf
    ADD(EAX, -0x16), //  add eax,-0x16
    POP(FLAGS), //  popf
    JMP(NOTEQUALS, 0x05), //  jne short 5
    MOV(EAX, 5), //  mov eax,05
    CMP(EAX, 0x37), //  cmp eax,37
},

Also the GUI can look than if there is a new github Release for auto updating the modder

[LitzeYT]

@LitzeYT how exactly you apply your changes, do you do it by IDA or Hexeditor? my second suggestion was a application that do the changes with byte changes on the orginal exe without everytime use a new one. So you can set memory adresses and the changes from there like in the Crusader Modding Comunnity the UCP.

new BinHook(6)
{
    CMP(EAX, 5), //  cmp eax,05
    PUSH(FLAGS), //  pushf
    ADD(EAX, -0x16), //  add eax,-0x16
    POP(FLAGS), //  popf
    JMP(NOTEQUALS, 0x05), //  jne short 5
    MOV(EAX, 5), //  mov eax,05
    CMP(EAX, 0x37), //  cmp eax,37
},

Also the GUI can look than if there is a new github Release for auto updating the modder

Basic things like the settlers number are permanently patched, the game speed is patched via the launcher.

I use different things, but mainly IDA. It was already in the idea to let the launcher patch everything.

[PodeCaradox]

Which language are you using for the Launcher? the permanently patches work with replace the exe or do you just change the values inside of the exe?

[LitzeYT]

Which language are you using for the Launcher? the permanently patches work with replace the exe or do you just change the values inside of the exe?

Currently we use C#. However, the launcher is currently being rebuilt for a ranked implementation.

The values in the ExE will be changed and it will be released as an update.

For more information you can also add me on Discord.

Sc0uty#8434

[nyfrk]

I think i found an even simpler way for allowing more than 100 unit selections. We could actually fix the buffer overflow that leads to the crash by guarding ecx at S4_Main.exe+26041B. Never let it become greater than 99 in the first place. This fixes the crashes. No need for a custom vector.

So i added the following code at S4_Main.exe+26041B:

MOV ecx,[S4_Main.exe+105752C]
CMP ecx,100
JL ok
MOV [S4_Main.exe+105752C],100
ok:

No crashes so far. However there are no health bubbles for the surplus units in the selection. Maybe we could draw them manually later. Or maybe we can look more deeper into the function and see why it needs that buffer at all.

my second suggestion was a application that do the changes with byte changes on the orginal exe without everytime use a new one.

Actually i was surprised with that too and expected a proxy DLL or at least a loader. Byte patching non ram mapped binaries is a pain. I would not do that.

Basic things like the settlers number are permanently patched, the game speed is patched via the launcher.

I'll suggest using a proxy DLL. This will add better compatibility to other mods and it will make enabling/disabling possible while the game is actually running. When proxying d3d9.dll we could also easily draw overlays onto the game (eg drawing additional health bubbles or additional menus xD).

Is it possible that you provide me with your patched version so I can have a closer look at it ? Then I could look at the other problems that are still to be done, like the ones you mentioned.

I am currently quiet busy and cannot spend too much time. Till the end of the week I will create you a simple visual studio C++ project that builds a DLL that can be injected into the process, that hooks into the game and applies the 100 units patches to the running game. If find the time i will make it a proxy dll of d3d9.dll so you do not have to inject it yourself.

[LitzeYT]

I think i found an even simpler way for allowing more than 100 unit selections. We could actually fix the buffer overflow that leads to the crash by guarding ecx at S4_Main.exe+26041B. Never let it become greater than 99 in the first place. This fixes the crashes. No need for a custom vector.

So i added the following code at S4_Main.exe+26041B:

MOV ecx,[S4_Main.exe+105752C]
CMP ecx,100
JL ok
MOV [S4_Main.exe+105752C],100
ok:

No crashes so far. However there are no health bubbles for the surplus units in the selection. Maybe we could draw them manually later. Or maybe we can look more deeper into the function and see why it needs that buffer at all.

my second suggestion was a application that do the changes with byte changes on the orginal exe without everytime use a new one.

Actually i was surprised with that too and expected a proxy DLL or at least a loader. Byte patching non ram mapped binaries is a pain. I would not do that.

Basic things like the settlers number are permanently patched, the game speed is patched via the launcher.

I'll suggest using a proxy DLL. This will add better compatibility to other mods and it will make enabling/disabling possible while the game is actually running. When proxying d3d9.dll we could also easily draw overlays onto the game (eg drawing additional health bubbles or additional menus xD).

Is it possible that you provide me with your patched version so I can have a closer look at it ? Then I could look at the other problems that are still to be done, like the ones you mentioned.

I am currently quiet busy and cannot spend too much time. Till the end of the week I will create you a simple visual studio C++ project that builds a DLL that can be injected into the process, that hooks into the game and applies the 100 units patches to the running game. If find the time i will make it a proxy dll of d3d9.dll so you do not have to inject it yourself.

That sounds good!

I'm going to patch it up and give it a try in my place.

Currently I see it so that it is perhaps not even necessary to display the life ad, if it does not make too much work, many would be already satisfied with the current state of the selection - more than 100 -.

Currently I'm working on a proxy DLL for the possibility to add gamemodes, for example certain units in the game to lock or "fun" gamemodes like out of 5 swords 10 soldiers come out. So far it works quite well with it.

That sounds even better, then I wait for the end of the week.

If you push it, I can take a look at it myself.

If you add me on Discord I can invite you to a group of 11 people, who all deal with Settlers IV patching, like HD patching, hotkeys or other things, if you have it.

Sc0uty#8434

[PodeCaradox]

injected into the process, that hooks into the game and applies the 100 units patches to the running game. If find the time i will make it a proxy dll of d3d9.dll so you do not have to inject it yourself. there is already a directx wrapper class on github where you can inject a dll into a process. https://github.com/elishacloud/dxwrapper it works very nice and can change directx behavior :) the other thing was maybe to help with a launcher with wpf and c# looks great and you can have easy update from github releases which im already written.

[nyfrk]

See here for an implementation of my my previously mentioned approach: https://github.com/nyfrk/S4_UnlimitedSelection

I also added a binary release so you can test it without compiling it yourself.

I have not made it a proxy DLL so you have to use an injector for now. However, I also provided one in the repo.

[LitzeYT]

See here for an implementation of my my previously mentioned approach: https://github.com/nyfrk/S4_UnlimitedSelection

I also added a binary release so you can test it without compiling it yourself.

I have not made it a proxy DLL so you have to use an injector for now. However, I also provided one in the repo.

Thank you very much.

I'll take a closer look. Am I allowed to take this and give it as a permanent injection ? For example, when it's launched from the launcher? Or in general Allows to integrate it in the patch ?

Of course with credits.

Best Regards LitZe

[LitzeYT]

See here for an implementation of my my previously mentioned approach: https://github.com/nyfrk/S4_UnlimitedSelection I also added a binary release so you can test it without compiling it yourself. I have not made it a proxy DLL so you have to use an injector for now. However, I also provided one in the repo.

Thank you very much.

I'll take a closer look. Am I allowed to take this and give it as a permanent injection ? For example, when it's launched from the launcher? Or in general Allows to integrate it in the patch ?

Of course with credits.

Best Regards LitZe

Im getting Crashes while i try to select 200 Units.

[LitzeYT]

See here for an implementation of my my previously mentioned approach: https://github.com/nyfrk/S4_UnlimitedSelection

I also added a binary release so you can test it without compiling it yourself.

I have not made it a proxy DLL so you have to use an injector for now. However, I also provided one in the repo.

I have a different checksum than you, even at a complete new installation.

[nyfrk]

I have a different checksum than you, even at a complete new installation.

That's probably because I am working on an older version.

Im getting Crashes while i try to select 200 Units.

I just clamped the health bubble loop but kept the selection marker loop. Therefore you could at least see what was selected (see image). For older versions this seems to work fine but it looks like it does not for more recent versions like yours. So we probably have to clamp them too which will remove them for the surplus units (>100).

I will push a fix soon so you can try whether clamping the selection markers will solve the crash for you.

There are probably other overflows somewhere which I did not investigate. I just fixed the fatal errors. You'll have to see whether unwanted effects occur and when you find one patch it too. For example the right click command seems to break when using it with too many selected units. That's why i just increased the limit to 250 units per selection which seems to work fine.

Am I allowed to take this and give it as a permanent injection ? For example, when it's launched from the launcher? Or in general Allows to integrate it in the patch ? Of course with credits.

I released it under MIT license so your use case is covered. You can use it with your own injector/patcher. However, I recommend keeping the DLL since it will be easier for you to merge with the upstream. But that decision is up to you.

[LitzeYT]

I have a different checksum than you, even at a complete new installation.

That's probably because I am working on an older version.

Im getting Crashes while i try to select 200 Units.

I just clamped the health bubble loop but kept the selection marker loop. Therefore you could at least see what was selected (see image). For older versions this seems to work fine but it looks like it does not for more recent versions like yours. So we probably have to clamp them too which will remove them for the surplus units (>100).

I will push a fix soon so you can try whether clamping the selection markers will solve the crash for you.

There are probably other overflows somewhere which I did not investigate. I just fixed the fatal errors. You'll have to see whether unwanted effects occur and when you find one patch it too. For example the right click command seems to break when using it with too many selected units. That's why i just increased the limit to 250 units per selection which seems to work fine.

Am I allowed to take this and give it as a permanent injection ? For example, when it's launched from the launcher? Or in general Allows to integrate it in the patch ? Of course with credits.

I released it under MIT license so your use case is covered. You can use it with your own injector/patcher. However, I recommend keeping the DLL since it will be easier for you to merge with the upstream. But that decision is up to you.

Maybe it's possible that you could give me your ExE pusht ? Well, I am currently trying it on the latest Settlers IV version (2.50.1516). Then maybe I can understand the problem better, if it is really the exe. I will also have a look at it and if necessary I will make a push request.

Thanks for your effort and the permission!

[LitzeYT]

I have a different checksum than you, even at a complete new installation.

That's probably because I am working on an older version.

Im getting Crashes while i try to select 200 Units.

I just clamped the health bubble loop but kept the selection marker loop. Therefore you could at least see what was selected (see image). For older versions this seems to work fine but it looks like it does not for more recent versions like yours. So we probably have to clamp them too which will remove them for the surplus units (>100).

I will push a fix soon so you can try whether clamping the selection markers will solve the crash for you.

There are probably other overflows somewhere which I did not investigate. I just fixed the fatal errors. You'll have to see whether unwanted effects occur and when you find one patch it too. For example the right click command seems to break when using it with too many selected units. That's why i just increased the limit to 250 units per selection which seems to work fine.

Am I allowed to take this and give it as a permanent injection ? For example, when it's launched from the launcher? Or in general Allows to integrate it in the patch ? Of course with credits.

I released it under MIT license so your use case is covered. You can use it with your own injector/patcher. However, I recommend keeping the DLL since it will be easier for you to merge with the upstream. But that decision is up to you.

Which version do you use? All those who have tried it with me now have a different checksum than yours, without the community patch. Sure it's for the HE and not for the gold version ?

C13883CBD796C614365AB2D670EAD561

[nyfrk]

Were you trying the latest version with the additional guard for the selection markers? That should fix it as the next hardcoded limit would be 474 units per selection iirc. I am not aware of any other limits but maybe i just haven't found them yet.

All those who have tried it with me now have a different checksum than yours, without the community patch.

Yes, because I am using an older version of the HE (2.50.1516.0). I do not plan to update anytime soon. It would be too much work for me to adjust all the stuff I already reverse engineered (its much more than the few patches I released for the selection limit mod). It would be simpler and faster if you just update the sigs if they fail. That should be quite easy though. Open a pull request here when you managed to get some working sigs. Although since you said it crashes it will most likely find at least some sigs (otherwise the DLL would just do nothing). What specific sig or patch is causing the trouble for you?

Btw this issue is getting too long and off topic. Please start a new issue to the corresponding project. That way other people can find them too. Be as specific as possible (adding a crash dump / stack trace is always a good idea)

[LitzeYT]

Whats about the Fix for the ModAPI ? Alot of Players whould love it to use it in Multiplayer Matches.