keatincf
commented
8 hours ago The reward for overall inspection success does take into account the weight of points inspected due to the initial state. The step reward for inspected points does not. If the initial inspected points are included as part of the step reward, the agent would be rewarded for having a lucky initial state, rather than for learning something.
There are issues with the termination check for the weighted inspection environment where it isn't using the inspection state from the current step, but from the previous step.
It also looks like the issue occurs in the WeightedSixDofInspectionEnv and the Petting Zoo version of those environments.
on env reset self.prev_weight_inspected is set to 0.0 but on each step, it is set to self.chief.inspection_points.get_total_weight_inspected() before taking any action. So any points seen on initialization will not be included in the points reward. So in the end, the points reward will be total_weight - initial_weight, often significantly less than the success threshold since it sees a lot of points at first.