extracting the steering angle, throttle position and brake position.

[tyluckyma]

Hi @ai-tor

Thanks very much for your script. I'm playing around it these days, and successfully extract the images. However, I met a problem on extract the steering angle, throttle position and brake position. Seems like the code wasn't able to extract those three values in my test. Could you offer any suggestions? BTW, I haven't modified any codes but added a button to activate the program (F4). This is because I wasn't able to loggin to the game in my version since your code directly run the script when loading, and the loading process just get stuck there.

float SLScenario::getVehicleThrottlePosition() {
    return getFloatValue(vehicle, 0x8D4);
}

//[0,1]
float SLScenario::getVehicleBrakePosition() {
    return getFloatValue(vehicle, 0x8D8);
}

//[-1,1]
float SLScenario::getVehicleSteeringAngle() {
    return getFloatValue(vehicle, 0x8CC)/-0.7; //TODO: Depends on vehicle?
}

To be a bit more specific, could you help me on the following questions?

1. You implemented the above code to extract the data. May I ask how did you get those 0x8D4, 0x8D8, and 0x8CC? Will it be different in other version of GTAV? Mine is the Reloaded crack version, probably the VER_1_0_335_2_NOSTEAM. How could I get those values in my test?

2. Seems like the way you extract image frame is a bit different from deepdrive. May I ask which one is more efficient? Yours or his?

3. Seems like there should be a conversion between the extract value and the Vjoy joystick controller. You have not implemented it right? Can I directly use the control setting from deepdrive of Craig?

4. To me, it is very inconvenient in testing GTAV scriptHookV commands, because I have to build the .asi and re-start the game, and cannot even debug it. What's worse, there is no detailed documentation in their Native Database. Do you have any suggestions on a more efficient debuging? I want to contribute to this program, but I don't have much experience on both cpp and GTA Modding.

5. When will your Reinforcement Learning version come out? I cannot wait to test and learn it.

[aitorzip]

Hi @tyluckyma !

It is great that you added the F4 button to activate the script. It is in fact something I wanted to add, so if you share the code I would be glad to merge it into the repository. I am sorry you had so much trouble loading it.

I will answer your questions one by one:

1. Those codes are offsets of the game memory relative to the internal vehicle structure that should point to the correct memory addresses of the fields we want to get. To get the values I used CheatEngine, which is a tool that allows you to see into the memory registers of a running game. Took a time, but the process is something like:
2. Load GTAV into CheatEngine
3. Push the throttle button in the keyboard
4. Look for changed fields in CheatEngine and select those that are suspicious of storing the throttle value
5. Go back to two until you are sure you found the field of the throttle
6. Repeat the process for brake and steering

So as you see, it is an entertaining activity of reverse engineering. Unfortunately, this offsets change from game version to version. I only searched for the ones in my version, which is the last one (VER_1_0_877_1_NOSTEAM I would say). If you can I recommend updating or getting a newer version of the game, if you can't you could try this codes:

Steering Angle: 0x8AC or 0x89C Throttle: 0x8B4 or 0x8A4 Brake: 0x8B8 or 0x8A8

If none of this works, you will need to find the codes yourself with CheatEngine (http://www.cheatengine.org/), be careful with the installation, it includes lots of ransomware. If you undergo this adventure feel free to ask me anything whenever you need it :)

2. Yes! If you are asking on how are we capturing the pixels buffer I would say mine is more efficient. Craig is using an external program (OBS) to capture the frames, while I am using calls to the Windows library directly from the game script. If you are asking on how are we storing the images into the hard drive, I believe Craig's way is more efficient, he uses H5 format, which is very common in Data Science, while I use normal PNG files plus an index file. For both storing and loading Craig's way is more efficient, loading time may not be an issue taking into account that training time usually takes much more time, so it may be less than 1% of your model training time. However I was concerned about storing time, as it could be a bottleneck for the capture framerate. I run it in my crappy laptop and I was able to store at 10 Hz. So although mine is a less efficient way, it doesn't influence the performance.

3. I think so, especially for the steering angle. I don't plan to use Vjoy as I plan to keep external programs to zero, so the script is self-contained. To control the vehicle from an external program (something you will need for RL) I am developing a simple UDP interface to send the commands directly to the ScriptHook plugin. More soon :)

4. Yeah... Suffered the same, debugging ScriptHookV scripts is hell. The problem is that if your script crashes then you can't do anything about it except restarting the game, so an important thing is to keep things simple and do incremental developments. To debug variable values and so I recommend using two simple but effective functions included in the library:

void set_status_text(std::string str, DWORD time = 2500, bool isGxtEntry = false);
void update_status_text();

You can do something like:

set_status_text("Variable a: " + std::to_string(a))
update_status_text();

And your a variable will appear in the center of the screen during the game. I didn't have any experience in GTA modding when I started the project, but in the end you actually get very familiar with the functions in the native database. Also, don't hesitate to ask me anything about its methods, I can probably give you a good explanation of any function related to Player, Vehicle, Gameplay and Paths.

5. I am also enrolled in an Udacity nanodegree, so this very much depends on the projects I have to deliver. Hopefully I will have something usable for the end of this month. I will let you know! :)

Thank you very much for your feedback!

[tyluckyma]

Thanks for your quick response!!!

Your other two suggested groups of offset values are also not working in my version. I've also tried to find your version of GTAV, but the download speed is killing (10k/s~50k/s). So I decided to try out CheatEngin. It feels quite cool, but it takes a while to search for the values even though I get 16GB memory and 4GHz CPU.

May I ask how you managed to search the values while out of the GTAV window? I mean when you shift to the CheatEngin window, the key (W,A,S,D) to control car is released right? At that moment, the throttle/steer/brake values are supposed to be 0, right? So are you searching the "unchanged value compared to the first search"? And how you find that the throttle/steer/brake address is the vehicle address + offsets, did you used the pointer scan?

In addition, after trying out the CheatEngin, I realized that searching and directly modifying memory value to control the game can be dangerous. In this case, using Vjoy as Craig did might be safer. In fact, I carelessly modified a memory value which cause a problem to the System Boot Manager and the system disk (My C:) became inactivated. Luckily I used DiskGenius and WinPE to recover from that.

My F4 button is actually integrated from the sample code of NavieTrainer.asi in the ScriptHookV develop version. It's quite simple but uses several while true and may not be that efficient. I'll clean it a bit and send to you later.

[aitorzip]

Hi again! :)

I am sorry the offsets didn't work. So what I did was to put the GTAV window at one side of the screen and the CheatEngine in the other, so I played GTAV while seeing the values in CheatEngine. To avoid this issue of the searching time I did a small trick, here it goes:

The scripthook library includes a super useful method that tells yous the root memory address of an entity, it is getScriptHandleBaseAddress(int handle), look for it in main.h.

So you can print the address of your vehicle in the screen just as I told you in the previous message and then configure CheatEngine to search only for addresses between the vehicle root address and let's say 0x999 more (look for float values). Then just play and see the values changing, keep scrolling down until you find values that change according to your commands. Take into account that for example the Throttle value can have more than one address value, i.e., one value for the command input, and another for the actual throttle of the vehicle, although they are essentially the same, the first one will only work when you push the throttle button, but not when the throttle comes from the game's AI.

Well, do not change values in memory ever again for your own health! Actually I never do that, what I do to send commands to GTAV without a controller is to use the following method:

void RLScenario::performActions(float throttle, float brake, float steering) {
    CONTROLS::_SET_CONTROL_NORMAL(27, 71, throttle); //[0,1]
    CONTROLS::_SET_CONTROL_NORMAL(27, 72, brake); //[0,1];
    CONTROLS::_SET_CONTROL_NORMAL(27, 59, steering); //[-1,1]
}

Then, the throttle, brake and steering arguments can come from wherever you want, in my case I plan to put a listener UDP socket, so the self-driving agent in Python can connect to it and send the commands to the game.

Hope this helps!

[tyluckyma]

Sounds cool! I'm still on your SL parts of code and haven't reached your RL parts yet. But this way of control is apparently cleaner and clearer. Then the only thing left seems to be the initial position and the inconvenient debugging process. Have you thought about the ScriptHookDotNet environment? It seems that that environment supports plug and play of GTAV mod scripts.

But you really hook me up with your new version of DeepGTAV environment. The implementation from Python on the driving agent could provide tons of convenience! Hope it come out soon!

[aitorzip]

Yes! The initial position is something to solve, sometimes it just starts at very inconvenient places such as airports or beaches. If you know a good location to start you always can set it up in the config.ini file.

I have thought a lot about ScriptHook.NET, it also includes some really useful methods to get nearby pedestrians or vehicles. Although the same can be achieved with the C++ version, it takes longer. I ended up using C++ because I wanted to have good performance.

Yeah! I expect to have it by the end of this month! :)

[tyluckyma]

Hoops, eventually got my parameters~ Thanks~~

My offset for throttle&brake is 0x8A4, and steering is 0x89C. Yes, you've suggested already. What I found is that getScriptHandleBaseAddress(), in my case, is not returning the proper address value. The address returned by getScriptHandleBaseAddress is an 8 hex (like 0x54151ab2) and my CheatEngin cannot find that address.

Later I notice most of my GTAV5 address are 10-12 hex numbers, and I think probably there is something I'm not doing right. Therefore I go to check some Mods' source code and found some clues of using reinterpret_cast<uintptr_t>. Fortunately, in this way, I finally was able to print the 11 hex address (like 0x123456789ab) of the vehicle. After that, I used your trick and found 8A4 and 89C. (^_^). What's more, in my case, 0x8A4 can handle both throttle and brake, like [-1, 0] is brake and [0, 1] is throttle. That's a bit better compared to three values. Or do you have anyother reasons of using three values?

On the other hand, I realized that the control command is not linearly correlated with the car movements. For example, if I input throttle = 1, what the car perform is something like 0.95, and if I input 0.5, it becomes 0.2 or something. I guess the game is trying to make the control more realistic so that different car, different roads or driving environments will have different sensitivities on the control command. As a result, there probably have two kinds of offsets. One is the offset of the address, and the other is the offset of the values. So any idea on fixing the latter problem?

[aitorzip]

I am happy you succeeded! It is not an easy task! :)

Yes, exactly, there is an offset for the control input (A) and an offset for the "real" car command (B), so I guess each car will have its own map from A to B. I think it would be better to use B directly for both input and output, because then an agent trained with these values could work for any vehicle in GTAV, otherwise the behavior of the vehicle will change from car to car.

If we go for the A option for both reading and writing, there is no way we can directly read the vehicle throttle while the AI is driving and would need to discover this non-linearity from B to A for each vehicle. This can be done with RL, because you can observe the behavior of the vehicle, but not with SL.

Currently I am reading the values from B and inputing through A (performActions method) so this is not good, because then an agent trained with the values from B will behave differently when testing it.

I still have not developed any agent so I don't know if this system will work. If it doesn't maybe we could try something like this, which would allow us to use always B:

float RLScenario::setVehicleThrottlePosition(float throttle) {
    return setFloatValue(vehicle, 0x8D4, throttle);
}

And hopefully the car will move :)

I will do some checks for the December release and see which approach is better! Thank you very much for your feedback! :D

[tyluckyma]

Hi @ai-tor

Tried my health another time~~ Fortunately found something useful. The steer value can directly be modified from memory, while the throttle and brake not. In this case, at least the steering value can be accurately controled by the learning agent. To the other two values, I've tried to look at the input-output relationships. I've found that actually the input value is linearly related to the memory value. In my version, memThrottle = (4/3)*controlThrottle - 1/3. Brake is same (only when you are drifting, it's a bit different).

After all, game is game, they still using simple equations to simulate the relationships. (^_^)

[aitorzip]

This is great!! We can use this relations to have accurate control. What happens exactly when you try to modify the values of throttle and brake in memory?

[tyluckyma]

As you mentioned, I also found two values are suspicious as the throttle address. Both of them works when AI is driving. I've tried to modify both of them but nothing happened. Yeah, nothing. I used while-true to consistently control them. However, only the steer value is working properly.

On the other hand, it's also possible that it can work just as the steer, and I think it's quite possible. Either the address is not right or I had some bugs in my test.

[aitorzip]

Great! I will do some tests and integrate it into the project, I will also add you at the credits section of the Readme :)

For now I will close the issue!

[saravanabalagi]

I'm on Reloaded No Steam 1.0.350.1. What is the minimum version to get steering, throttle and brake working? I'm getting zeros in those places as well. Is there a workaround?

[saravanabalagi]

Updated my game to 1.41 (1.0.1180.2) and no more zeros... :)

[wujiyoung]

I have two question:

1. I want the vehicle brake to stop, setFloatValue works fine, while _SET_CONTROL_NORMAL makes the vehicle brake to moving reversely, does _SET_CONTROL_NORMAL can make the vehicle brake to stop?
2. I set the brake value with both _SET_CONTROL_NORMAL and setFloatValue, but when I get the brake value with getFloatValue and GET_CONTROL_NORMAL, both return zero.

Does anyone know how this happen?