weiyx16
commented
5 years ago The problem is caused by this:
- First, the input of our model is a cropped image (128 X 128) around highest point in the Affordance Map, which has size of 424 X 512.
- So if the point is near the boundary of the Affordance map, it will cause the cropped image fail to fit 128 X 128.
- In your case, the maximium point is [370, 335], so in one side, there's only 424-370=54 left, which is added with other side 64 pixel to be 118.
Solution
Our solution for this situation is to add boundary around the Affordance Map, i.e. ref to this line, it is the boundary of the collecting box in the simulation. And I postprocess the Affordance Map to set point value out of the boundary to be zero , so the highest point won't be too close to the boundary. You can shrink it to a safer size, like [64, 424-64, 64, 512-64]
I know it's a naive solution, but I don't think the way to pad it with 1 or 0 will be better.
yangshuo11
Hello, Wei, when I am training the agent in the V-rep simulator, there will always occur a ValueError of "inconsistent shape (x, 128, 3) with (128, 128, 3)". Do you have any idea with this issue? Part of the log is as follows:
-- Maximum Location at [210 227] -- Metric for current frame: 0.734154 With peak_dis: 0.173634, flatten: 0.820850, max_value: 0.924716 13478it [15:28, 9.80s/it] -- Push from [152. 198.] to [106.74516600406096, 243.25483399593904]
-- Maximum Location at [180 340] -- Metric for current frame: 0.769855 With peak_dis: 0.264738, flatten: 0.859865, max_value: 0.852448 13479it [15:38, 9.88s/it] -- Push from [193. 311.] to [257.0, 311.0]
-- Maximum Location at [175 349] -- Metric for current frame: 0.880095 With peak_dis: 1.000000, flatten: 0.890577, max_value: 0.785750 [!!] Nearest distance: 0.084278 [*] Update the scene 16 with 3 object removed
-- Maximum Location at [175 349] -- Metric for current frame: 0.880095 With peak_dis: 1.000000, flatten: 0.890577, max_value: 0.785750 13480it [15:57, 12.57s/it] -- Push from [146. 398.] to [210.0, 398.0]
-- Maximum Location at [370 335] Traceback (most recent call last): File "main.py", line 88, in
tf.app.run()
File "/home/ys/anaconda3/envs/active-perception/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "main.py", line 72, in main
agent.train()
File "/home/ys/Active-Perception/DQN/dqn/agent_18.py", line 66, in train
screen, reward, terminal = self.env.act(action, if_train=True)
File "/home/ys/Active-Perception/DQN/simulation/environment.py", line 517, in act
self.screen, self.local_afford_new, self.location_2d = self.camera.local_patch(self.index, (self.screen_height, self.screen_width))
File "/home/ys/Active-Perception/DQN/simulation/environment.py", line 222, in local_patch
return get_patch(location_2d, self.cur_color, self.cur_depth, post_afford, patch_size)
File "/home/ys/Active-Perception/DQN/util/utils.py", line 44, in get_patch
patch_rgbd[:,:,0:3] = patch_color
ValueError: could not broadcast input array from shape (118,128,3) into shape (128,128,3)