search

kindredresearch / SenseAct

SenseAct: A computational framework for developing real-world robot learning tasks
https://www.kindred.ai/SenseAct
BSD 3-Clause "New" or "Revised" License
212 stars 42 forks source link

Warning: incomplete packet from UR #47

Open PorkPy opened 5 years ago

PorkPy commented 5 years ago

So I can get my UR5 robot moving using the example reacher scripts and also the pre-trained model python3 examples/advanced/ur5_reacher.py ./examples/advanced/pre_trained_models/ur_reacher_2_trpo.pkl

The problem is; the robot makes the same movement repeatedly. There is no difference between the untrained and the pre-trained models. In both situations, I get a recurring error saying Warning: incomplete packet from UR I suspect this is the joint feedback to SenseAct, so the agent doesn't know where it is maybe?? I'm using URSoftware 3.8.0.61336. I will try to downgrade my URSoftware to 3.3.4.310 and retest.

I have now downgraded my UR software to 3.3.4.310 and it is working brilliantly! I'll put a video on Twitter ATdommckean. I do still have some questions though. In the console output, what is meant by; Hiccup of 1.48ms overhead between UR packets) ? and; WARNING:root:Agent has over-run its allocated dt, it has been 0.4206395149230957 since the last observation, 0.3806395149230957 more than allowed Should I be concened?

Thanks again for this awesome package.

armahmood commented 5 years ago

Congratulations on getting the learned model work on UR5!

Were you also able to get learning from scratch?

A hiccup in the UR communicator means the time between two UR packets, which should have been close to 8ms (Figure 3, Mahmood et al. 2018), was irregular. This can happen while communicating wirelessly with the URControl or when the CPU cores are too busy to let sensor_handler process data quickly among other reasons.

The cycle time of agent-environment interaction or dt may over-run again due to resource contention or making expensive learning updates synchronously within an episode.

They are not of supreme concern if they occur once in a while, for example, once in every five minutes or so.

Looking forward watching the videos!

PorkPy commented 5 years ago

Hi @armahmood , Thanks for your reply. Despite using the 'learned' model, there was still some learning to do. The prescribed 150,000 timesteps in 'ur5_reacher.py' which took about 3 hours did tune the policy, in fact, it was doing quite well after about 100,000 timesteps. Of course, I should do my own experiments from scratch and when I do I'll let you know how it goes. The warnings I am getting occur more than every 5 minutes. Maybe this could be solved by changing the process priority levels??. Also, I'm using a GPU for learning, so surely the cpu can just concentrate on communications??

Anyway, I have a new problem now. I've just been told that I'm losing my ur5 robot, however, I do have access to two UR10s, but I will need to somehow alter this code to work on them. Have you done any work with this code on a UR10? Could you perhaps point me to which snippets of code I will need to alter to get this working? That would be a massive help. Thanks, Dom.

PorkPy commented 5 years ago

Hi @armahmood ,

I don't think I have been using the trained policy after all. I don't think the file exists! So the line I used to start the script; python3 examples/advanced/ur5_reacher.py ./examples/advanced/pre_trained_models/ur_reacher_2_trpo.pkl was only running ur5_reacher.py, so it was learning from scratch to begin with! lol. Explains why it took a couple of hours to converge, still, I don't think a couple of hours is too long to wait. I was expecting to wait much longer. Is there a copy of the learned policy anywhere please?

Thanks, Dom.

PorkPy commented 5 years ago

Hi again @armahmood , I have a problem. To get either the UR5 or UR10 working correctly I need to downgrade the UR software. However, by downgrading I can no longer use the gripper as the Robotiq software is not compatible with earlier versions of UR software. Also, other people need to use these robots and so running them on old software is not going to work long term. Can you tell me what I need to change in the code in order to use it with updated versions of the UR software please? I'm currently looking through UR_communicator and reacher_env to see if I can find anything obvious. I hopw we can find a solution.

Many thanks, Dom.

PorkPy commented 5 years ago

I've found the UR10 pull req. However, I'm still getting 'Warning: incomplete packet from UR' The REALTIME_COMM_PACKET_SIZE = 1108. Is this right for polyscope 3.5? There are also two additional names in the packet list ''elbow_position', '>f8', (3,)), and ('elbow_velocity', '>f8', (3,)' Are these right? Thanks, Dom.

nily-dti commented 5 years ago

@PorkPy if you haven’t found a solution yet, then have a look at this: https://www.universal-robots.com/how-tos-and-faqs/how-to/ur-how-tos/remote-control-via-tcpip-16496/

SenseAct connects through TCP socket to the URs real-time (RT) communication interface which is specified in the excel document at the above resource (direct link).

According to this, the packet size is correct. And yes, the elbow position and velocity were added in UR v.3.5

nily-dti commented 5 years ago

@PorkPy regarding your questions on using the pretrained model. Are you using PR #44?

ghx-0228 commented 5 years ago

@PorkPy I have a simple question for the UR projects.

  1. I want to reproduce the result shown in the video from scratch. I set the host address and am able to run the code. But the real UR can not move at all and I just obtained the plot like this. The red node is static at origin while the blue node repeats appearing and disappearing at the same position. image Additionally, I encountered the same problems as yours. "warning: incomplete packet from UR" and " hiccup of time between UR packets" Have you solved them and could you help me solve it? Thanks very much!
nily-dti commented 5 years ago

Which version of the UR are you using? In my research group we’ve replicated the work presented in the paper. You can consult our implementations of the experiments and our fork of the SenseAct framework.

ghx-0228 commented 5 years ago

@nily-dti Thanks for your suggestions. UR 5. and Polyscope 3.5

ghx-0228 commented 5 years ago

@nily-dti thanks very very much for your help. It works!!! But I want to enquire that what is the difference between your modified code and original code. Why do you need Docker to implement this task?

nily-dti commented 5 years ago

@ghx-0228 no problem. Glad I could help :-)

The main difference between the original SenseAct and our forked version is that we’ve incorporated PR #29 by @Olimoyo and added support for Docker. Most of the work we’ve performed in relations to SenseAct is to replicate their experiments.

Docker is not strictly needed to conduct the experiments, but helps avoid the “dependency hell”. Thus we’re sure that researchers attempting to reproduce our work can do it using the same software setup.

Hope it helps :-)