Not working repo

[KamalMokhtar]

Dear Xu,

This project does not work, there are many bugs and issues, I have tried to reproduce your work for 3 months now, I am not able to get your results. I hope you can reproduce and share the results.

Hoping to hear from you! Kamal

[xukechun]

Hi Kamal:

1. What means "many bugs and issues?" Does anything confuse you? Or any command confuses you?
2. I can share our [reproduced model] tomorrow.
3. Also, I mentioned in the readme that we found that exploring with a pre-trained goal-agnostic grasp model can effectively speed up the training process. You can try this technique.

[Kamalnl92]

Hi xukechun,

1. I will make a report and show all the problems. I should have it done by tomorrow.
2. That would be great, can you share all the models and how many episodes or iterations you have trained it for. I can create a shared folder so that you can place all the model's pictures (logs). that will give me more insight into where things go wrong.

Thank you!

[Kamalnl92]

Hi xukechun,

• Problem 1)

This happens after so many training iterations at any stage usually between the 200 and 2000 iterations.

The above two problems are related I believe I solved this issue. https://github.com/xukechun/Efficient_goal-oriented_push-grasping_synergy/issues/3

The training stops when this RunTime error appears.

• Problems 2)

• Problem 3)

• Problem 4)

• Problem 5)

This project is quite unstable (not working), I would like to cooperate to solve all the problems, if you would like to know more (i.e. print statements in the source code) I would gladly do that.

I would like to see the models and the terminal output and log/ files during training from your side. Could you upload them to the below link? that would assist me in seeing the difference.

https://drive.google.com/drive/folders/1-hluT4AHMCYn0QHmrFC50WSVdIAWKwOF?usp=sharing

[xukechun]

Thanks for your problem report. I think you can refer to the repo of VPG to solve most of your problems:

• Problem 2 corresponds to this issue and Problem 3 corresponds to this issue.

• For problem 4 and problem 5, why not use "try and expect" to fix the bugs? I find all the relative codes of your problems are those we follow VPG, so I strongly recommend you to refer to that repo.

• I provide our intermediate models grasp model and push-grasp model for you, you can train stage two and stage three upon them respectively. Also, our second model can reach average performance of 97.8 86.7 3.72 for the 10 cases we mentioned in readme, which is similar to the results of our paper. You can train stage three upon it.

• Our reproduced models are trained with torch 1.2.0 with Cuda 10.2 and 2080Ti.

[Kamalnl92]

Thank you for your reply.

• Problem 2 and 3, that mean someone has to be attending all the time in case the problem occurs to close the Dynamic content window on time. Thas seem not plausible?

• Problem 4 It seems like the attempt is to fix the symptoms, why not the cause of the problem, what causes it? is it permitted that move_direction[0] or/and move_step[0] having the value '0'

• Problem 5 you linked the repo https://github.com/andyzeng/visual-pushing-grasping . Do you recommend I should not use your code?

• Thank you for the model provided could you provide the other models as well. could indicate for how many iterations you trained the models?

[xukechun]

• For problem 2 and 3, it just needs three times to close. Please see the issue comments carefully.
• For problem 4 and 5, I recommend you to fix the bugs to find why it happens. Because I can't reproduce your problem. Also, I link the repo of VPG just to demonstrate that those codes are referred to VPG and we just follow them, but not recommend you not use our code. I just state that we follow the former open source work which is followed many times. So these referred codes are more likely to work normally.
• I recommend you to solve problem 4 and 5 by "try and expect" or setting breakpoints to see what's wrong during the process since I can't reproduce your problems. What you need to do is to be patient to find what's wrong :)

[Kamalnl92]

• The problem does not happen instantly, besides I am running CoppeliaSim in headless mode, because I am using an HPC cluster, with ubuntu 18.04 and all the dependencies requirement as described in the readme. Please I would like you to know that I checked everything carefully, as I mentioned before I have been trying to reproduce your work for 3 months now.
• I have already used try-except and the performance of the model eventually after stage 3 training has not come close to what is described in the paper.

I believe if you reproduce the work by using the code in the GitHub repo here, I am sure you would face the same problems, while training I would appreciate it if you can save the terminal output, which can be easily done by ' > terminaloutput.txt'. That will save the output of the terminal while training. Als the models that have been trained, again you may use the shared folder I have created above to upload everything including the log/.

I hope you can do that. That would make things clear for everyone. Thank you.

[xukechun]

Sorry I haven't try the headless mode before. The solution of problem 3 and 4 might need a display window since you should close the popup window for three times.

[Kamalnl92]

I understand, Would it be possible for you to use this repo code and train the models again please, I would appreciate that.

• models
• terminal output saved in txt file
• log/ directory

[Kamalnl92]

Another issue next to the above-mentioned issues (problem 6).

The mask of the q values are not fitting the objects, see the image below, this example happened while training the grasp model 554 and 556

[Kamalnl92]

Problem 7, pushes are just done at nowhere. Iteration 27 push net

[Kamalnl92]

Hello,

Please do not close the issue when it is an open issue (not solved) your project should be reproducible, and so far this is not the case.

[Kamalnl92]

I see you have closed the issue, I have noticed that the masking is not working, maybe that is the cause of some of the problems.

[xukechun]

OH. So sorry for my mistaken close. Do you mean the mask as input is not necessary?

[xukechun]

Another issue next to the above-mentioned issues (problem 6).

The mask of the q values are not fitting the objects, see the image below, this example happened while training the grasp model 554 and 556

Is that this issue? Is there any report about this problem? I will check the code of this repo carefully for the mask generation.

[Kamalnl92]

The mask is necessary! I believe if you run it you will see that it's sometimes not working, as you see in the image it was for iteration 556, so it does not happen at the first iteration. I am not sure what is helpful for you from my side.

[xukechun]

Thanks for your comment! I'll run and check carefully.

[Kamalnl92]

Thank you for your reply!

[xukechun]

Hi, we just reproduce the mask problem and correct it. We have modified the function get_obj_mask and get_obj_masks in robot.py to the same as get_test_obj_mask and get_test_obj_mask . We feel really sorry for this negligence when reorganizing codes. The key codes are these two lines: https://github.com/xukechun/Efficient_goal-oriented_push-grasping_synergy/blob/feb5dd8196e60f2e454ecf2a26c438043eb65ab1/robot.py#L236-L237 Also, another modification in main.py is to make push action more stable. Hope my reply will help you and again sorry for our negligence.

[Kamalnl92]

Thank you, I will check it :)