The status of the drone is grounded

[robot-xyh]

Hi When I use your program for training, it will happen that the drone will land on the roof. At this time, the drone will no longer move or restart. The program has been waiting。 I use the debugging interface of airsim and it shows is grounded

The command line window always displays information at the end：

----------------------------------------------------------------------------------------
episode:454, reward: 1.4655585290142596, mean reward: -0.22, score: -7.5264491171413965, epsilon: 0.5605743913342761, total steps: 15292
Total Memory: 10.0 GB
Allocated Memory: 209.94 MB
Cached Memory: 234.0 MB
Free Memory: 9.56 GB
Episode is done, episode time:  138.60098004341125

I found the isgrounded function in the Microsoft program, at AirSim/AirLib/include/physics/FastPhysicsEngine.hpp，in line 92. But this API is not provided in python. Have you ever encountered this situation?

[karascr]

Hi I have never encountered in this project. I did not handle the landing state because in our simulation environment, we did not want our drone to land so we ended the episode by reward function if the drone goes too low. But in my previous project I had something like that so you can try this: below this line in env.py reward function diff = self.last_dist - dist if diff results as 0.0 you can assume drone is stuck and you can set done = 1. After that program will end the episode and start the next one. I trained with a higher "ClockSpeed" setting, so you may need to adjust time.sleep(.4) in step function because I realized that drone moves too quickly in normal clock speed. Btw I updated some old codes about prioritized memory.

[robot-xyh]

thank you for your reply When the drone is stuck, I use the "client.getMultirotorState().landed_state" function to get the drone status display. It is not landed at this time, and the number of collisions on the debug interface does not increase. I have tried to increase the speed judgment in the step function. If the speed is less than 0 for a long time, then set done=1, but it doesn't seem to work.

        speed_stop=np.linalg.norm([quad_vel.x_val, quad_vel.y_val, quad_vel.z_val])
        global speed_zero_count
        if speed_stop<0.1:
            speed_zero_count+=1
        if speed_zero_count==5:
            done=1
            speed_zero_count=0

I will try your way.thank you

[robot-xyh]

Hi I tried to add position judgment to the function of compute_reward.But it still doesn't work.

def compute_reward(self, quad_state, quad_vel, collision):
        """Compute reward"""
----------------
            if dist < 10:
                reward = 500
            else:
                reward += diff
            if diff==0:
                done=1
-----------------

I added a print speed statement in the step function, When the drone is stuck , the program did not enter this step. Because the speed is not printed. However, when the drone is not stuck, the program can correctly output the number of times the speed is 0

def step(self, action):

        self.quad_offset = self.interpret_action(action)
---------------------------------------------------
        quad_vel = self.client.getMultirotorState().kinematics_estimated.linear_velocity
        speed_stop=np.linalg.norm([quad_vel.x_val, quad_vel.y_val, quad_vel.z_val])
        global speed_zero_count
        if speed_stop<0.1:
            speed_zero_count+=1
            print("__________________________________")
            print(speed_zero_count)
        if speed_zero_count==10:
            speed_zero_count=0
            #done=1
---------------------------------------------

This is really weird. Then I used the client.getMultirotorState().landed_state statement to determine whether it landed, and I got False. After about two hours, airsim will reset again, I think it’s probably timeouttriggered The functions I use are agent.py and Neighborhood environment. The height of 7m is just easy to get stuck on the roof.

I read the source code of airsim and found that the collision information contains such a comment: // make it stick to the ground until the magnitude of net external force on body exceeds its weight in line 338 https://github.com/microsoft/AirSim/blob/238c8e708dab831bf5835b1c954abfb0136e9277/AirLib/include/physics/FastPhysicsEngine.hpp

I printed the log and found that the collision count did not increase, but another number collision#*** with Roof_Apex_9 kept increasing. I found the api in the picture, in line 115 https://github.com/microsoft/AirSim/blob/416de29f3d2034d90bff5d59eb017db9edf2b777/Unreal/Plugins/AirSim/Source/Vehicles/Multirotor/MultirotorPawnSimApi.cpp

[karascr]

Hi It is really weird, I wanted to try client.getMultirotorState().landed_state statement but last week I sent my laptop to service, I will notify you after trying it, when I get my laptop back. As I understand from this; I added a print speed statement in the step function, When the drone is stuck, the program did not enter this step. Because the speed is not printed. I think the problem is related to the environment or unreal engine because step function must be called even if the drone is stuck. I mean there should be a bug or something in the environment I guess. Because in our environment, when the drone is stuck, the step function was still working properly. Is this happens only with the roof?

[robot-xyh]

Yes, this situation only happens on the roof. The situation about grounded is mentioned in the program https://github.com/microsoft/AirSim/blob/6d20663e0818448613be0332009b02bd6341589e/AirLib/include/physics/FastPhysicsEngine.hpp in line 373

if (body.isGrounded()) {
            // this stops vehicle from vibrating while it is on the ground doing nothing.
            next.accelerations.angular = Vector3r::Zero();
            next.twist.linear = Vector3r::Zero();
            next.twist.angular = Vector3r::Zero();

in line 338

if (body.isGrounded()) {
            // make it stick to the ground until the magnitude of net external force on body exceeds its weight.
            float external_force_magnitude = body_wrench.force.squaredNorm();
            Vector3r weight = body.getMass() * body.getEnvironment().getState().gravity;
            float weight_magnitude = weight.squaredNorm();
            if (external_force_magnitude >= weight_magnitude)
            {
                //throttledLogOutput("*** Losing ground lock due to body_wrench " + VectorMath::toString(body_wrench.force), 0.1);
                body.setGrounded(false);
            }
            next_wrench.force = Vector3r::Zero();
            next_wrench.torque = Vector3r::Zero();
            next.accelerations.linear = Vector3r::Zero();

in line 21

FastPhysicsEngine(bool enable_ground_lock = true, Vector3r wind = Vector3r::Zero())
        : enable_ground_lock_(enable_ground_lock), wind_(wind)
    { 
    }

I think changing true to false should release the lock, but I haven’t tried it yet When the drone is stuck, I terminate the program, run takeoff.py in AirSim\PythonClient\multirotor, and find landed == airsim.LandedState.Landed is False, and then the environment will be automatically closed. I found client.getMultirotorState().landed_state Output does not containgrounded ，I personally think that landed and grounded are not the same.

[karascr]

Hi, sorry for the late reply I don't know if you already solve it but today it happened to me as well and I found a solution. I am storing the collided object id right after resetting the drone. self.last_collision_id = self.drone.simGetCollisionInfo().object_id Then in each step, I am checking if the id is changed or not, so if it changes, it means the drone has collided. has_collided= self.last_collision_id != self.drone.simGetCollisionInfo().object_id It works well for now.

[robot-xyh]

Thank you, I did not solve it, I modified the environment to avoid it from landing on the roof, thank you for your help