GTAV.exe crashing before finished loading

[MarkusKluge]

Supervised works perfectly for me. I have now trained a model with my own data and got the drive.py to run.

The only problem I have is, that in like 27 of 30 cases the gtav.exe is crashing (game still shows loading screen). On the server it says the following in the log:

Using TensorFlow backend.
Started server
GTAV connected
D:/DeepLearning/GTA V/VPilot-master/drive.py:64: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
  if (img == None): break
Sent commands array('f', [0.13982641696929932, -0.03970291092991829])
Received reward
0.9407272338867188
Sent commands array('f', [0.15126948058605194, -0.050635144114494324])
Received reward
0.9407272338867188

The amount of sent commands and received rewards is changing from try to try.

Only in like two cases it actually showed the game and spawned in a random location with vehicle. But after 20-40 seconds it also crashed and the vehicle did not accerlate (could be the models fault).

For me it feels like that the server is trying too early to send commands (before the game is even ready and the player is at a location within a car).

Any solutions?

[revilokeb]

@Mkmk93 I am seeing the same server log as you but the game never crashes, so your crashes seem to originate from somewhere else. What is your setup? Are you running the server and the game on the same machine / GPU? Are you making sure that you are having sufficient memory on GPU?

I can however confirm that also my vehicle does not accelerate based on my trained model so far which is somewhat surprising to me since the server is sending positive throttle (which it also seems to do btw when using Aitor's dummy model.h5). But anyway, I havent explored this yet in any detail...

[aitorzip]

Hi!

Sorry for the delay in the response.

First thing is that the vehicle doesn't start moving unless you have a throttle at around 0.8 I believe. This is something I am going to fix in a week, because using manual commands it doesn't need so much, so I think there is a mismatch there.

I think @revilokeb is right, your random crashes point to something external, probably lack of resources. Can you share how is your setup and also your current config.ini file?

Thanks!

[MarkusKluge]

I have a Windows 10 64-bit OS, 8GB Ram, GTX 980Ti (6GB).

I used the resolution 410x205 for training the model. I also tested it running GTA V on the same system and running on a seperate Laptop. In both times it works for like 1-5 images and then gtav.exe crashes.

I am gonna try a lower resolution now (for the model/training), maybe thats the reason. (Even though the drive.py isnt showing any OOM-Out of Ram issues)

My drive.py:

import argparse
import socket, struct
import numpy as np
import time
from array import array
from model import nanoAitorNet

class Server:
    def __init__(self, port=8000, image_size=(410,205)):
        print('Started server')
        self.image_size = image_size
        self.buffer_size = image_size[0]*image_size[1]*3;
        self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.s.bind(('192.168.178.22', port))
        self.s.listen(1)

        self.conn, self.addr = self.s.accept()
        print('GTAV connected')

    def recvImage(self):
        data = b""
        while len(data) < self.buffer_size:
            packet = self.conn.recv(self.buffer_size - len(data))
            if not packet: return None
            data += packet

        return np.resize(np.fromstring(data, dtype='uint8'), (self.image_size[1], self.image_size[0], 3)).astype('float32')

    def sendCommands(self, throttle, steering):     
        data = array('f', [throttle, steering])
        self.conn.sendall(data.tobytes())
        print('Sent commands', data)

    def recvReward(self):
        data = b""
        while len(data) < 4:
            packet = self.conn.recv(self.buffer_size - len(data))
            if not packet: return None
            data += packet

        print('Received reward')
        return struct.unpack('f', data)[0]

    def close(self):
        self.s.close()

if __name__ == '__main__':
    parser = argparse.ArgumentParser(description='Remote Driving')
    parser.add_argument('weights', type=str, help='Path to model weights')
    parser.add_argument('port', type=int, help='Port to listen to')
    parser.add_argument('width', type=int, help='Width of the image to receive')
    parser.add_argument('height', type=int, help='Height of the image to receive')
    args = parser.parse_args(['D:/DeepLearning/GTA V/VPilot-master/model.h5', '8000', '410', '205'])
    #args = parser.parse_args()

    aitorNet = nanoAitorNet()

    model = aitorNet.getModel(weights_path=args.weights)
    x = np.zeros((50, args.height, args.width, 3), dtype='float32')

    server = Server(port=args.port, image_size=(args.width, args.height))
    while 1:
        img = server.recvImage()
        if (img == None): break
        x = np.roll(x,-1, axis=0)
        x[-1] = img

        commands = model.predict(x[None,:,:,:,:], batch_size=1)
        server.sendCommands(commands[0,0], commands[0,1])
        reward = server.recvReward()
        if (reward == None): break
        print(reward)

    server.close()

config.ini

[common]
mode=1
imageWidth=410
imageHeight=205
car=0
weatherChangeDelay=600
initialWeather=-1
initialHour=7
initialMinute=-1
initialPosX=-83
initialPosY=58
maxDuration=9

[supervised]
setSpeed=40.0
drivingStyle=6
captureFreq=10
datasetDir=E:\GTAVDataset\

[reinforcement]
reward=1
desiredSpeed=15.0
desiredAgressivity=0.5
host=192.168.178.22
port=8000

[aitorzip]

Hi @Mkmk93

Some users have experienced weird errors depending on the size of the images. Try to set it to (320, 160) which I am sure it is working to see if the problem persists, if that fixes the error I will try to figure out why the change in resolution is crashing the game.

Thanks!

[MarkusKluge]

Hi @ai-tor

I have now tested it with 220x110 resolution and model. Finally no crash and the server is sending/receiving information. It is working - yeah!

It even is able to accerlate and steer a tiny bit until it crashes into the wall because of too less training :D

Thanks for your support and thanks for creating this awesome project btw! Now I can finally take time to create better data and actually train my model so it is able to drive.

I guess you can close this now.

[aitorzip]

Awesome! :)