microsoft / AirSim

Open source simulator for autonomous vehicles built on Unreal Engine / Unity, from Microsoft AI & Research
https://microsoft.github.io/AirSim/
Other
16.3k stars 4.53k forks source link

Nodisplay mode reinforcement learning training and test Image problem #1939

Closed EnderUPC closed 3 years ago

EnderUPC commented 5 years ago

Hello everyone,

I am working on deep reinforcement learning and I have been training the drone in Airsim for a while. Recently, I have encountered a problem. Whenever I train the drone in No-Display mode to reach a point in the NH environment, the test results with No-Display mode and without No-Display mode are not the same. In the test, the drone follows a very different trajectory and always crashes when the rendering is available. However, when the no-display mode is active, the drone reaches successfully the destination without any problem.

I have read the No-Display mode, and I think the recording and Airsim APIs remain active in this mode. I am using Unreal Engine 4.18 and the recent Airsim API for Windows.

Thank you for your comments.

` def getScreenDepthVis(self):

    responses = self.simGetImages([airsim.ImageRequest(0, airsim.ImageType.DepthPerspective,True,False)],vehicle_name="")
    img1d = np.array(responses[0].image_data_float, dtype=np.float)
    img1d = 255/np.maximum(np.ones(img1d.size), img1d)
    img2d = np.reshape(img1d, (responses[0].height, responses[0].width))

    image = np.invert(np.array(Image.fromarray(img2d.astype(np.uint8), mode='L')))

    factor = 10
    maxIntensity = 255.0 

    newImage1 = (maxIntensity)*(image/maxIntensity)**factor
    newImage1 = array(newImage1,dtype=uint8)

    small = cv2.resize(newImage1, (0,0), fx=0.39, fy=0.38)

    cut = small[20:40,:]

    return cut

`

jonyMarino commented 3 years ago

Closed due to age. Please feel free to open a new issue (referring to this one) if you still have this problem. Thanks!