Open RozenAstrayChen opened 4 years ago
Hello I have some problem which is ouccr in training section. I found that there is a certain chance that UR5 will get stuck after grabbing an object when I train model and the gripper is always tremble.
train script is use
python main.py --is_sim --push_rewards --experience_replay --explore_rate_decay --save_visualizations
and the screen print
Training iteration: 8126 Change detected: True (value: 281) Primitive confidence scores: 0.945526 (push), 1.194881 (grasp) Strategy: exploit (exploration probability: 0.100000) Action: grasp at (8, 152, 120) Executing: grasp at (-0.484000, 0.080000, 0.051003) Current reward: 1.000000 Future reward: 1.194881 Expected reward: 1.000000 + 0.500000 x 1.194881 = 1.597440 Training loss: 0.030402 Experience replay: iteration 7085 (surprise value: 1.742928) Training loss: 0.012920 Grasp successful: True Time elapsed: 5.005533 Training iteration: 8127 Change detected: True (value: 1609) Primitive confidence scores: 1.113359 (push), 1.614010 (grasp) Strategy: exploit (exploration probability: 0.100000) Action: grasp at (12, 104, 103) Executing: grasp at (-0.518000, -0.016000, 0.041099) Current reward: 1.000000 Future reward: 1.614010 Expected reward: 1.000000 + 0.500000 x 1.614010 = 1.807005 Training loss: 0.125703 Experience replay: iteration 2556 (surprise value: 0.628174) Grasp successful: False Training loss: 0.000300 Time elapsed: 4.954972 Training iteration: 8128 Change detected: True (value: 847) Primitive confidence scores: 1.120282 (push), 1.313959 (grasp) Strategy: exploit (exploration probability: 0.100000) Action: grasp at (4, 104, 136) Executing: grasp at (-0.452000, -0.016000, 0.041861) Current reward: 0.000000 Future reward: 1.313959 Expected reward: 0.000000 + 0.500000 x 1.313959 = 0.656979 Training loss: 0.144055 Experience replay: iteration 1956 (surprise value: 0.274301) Training loss: 0.025816 ################################### ...deadlock happen...
v-rep screen
same problem
Problem describe
Hello I have some problem which is ouccr in training section. I found that there is a certain chance that UR5 will get stuck after grabbing an object when I train model and the gripper is always tremble.
train script is use
and the screen print
v-rep screen
System