a issue about sim2real

[a2821952]

Hello, I try to excute whole_body_controller_example in the pybullet and it works well. When I use whole_body_controller_robot_example in a real A1 robot with MPC, I found that the robot jumped up and fell down abruptly, is there any setting I have missed?

[erwincoumans]

Can you update and try again? We made some improvements, it should work better now. Note that the sim and real example is merged into one. First make sure the simulation works:

python3 motion_imitation/examples/whole_body_controller_example.py --show_gui=True --max_time_secs=2

Once that works, add the additional argument (disable show_gui, but it should work enabled as well)

python3 motion_imitation/examples/whole_body_controller_example.py --show_gui=False --max_time_secs=2 --use_real_robot=True

[a2821952]

Thank you for your advice, I try to update code and it works well in simulation. But in the experiment the robot was moving so fast that it could't keep its balance, so I tried to adjust some parameters such as time_step, action_repeat and time.sleep(0.0008) but falied to solve the problem of violent movement. Have you encountered any similar problems in A1 robot? Thank you for your patience :)

[erwincoumans]

All works well on the A1 here. If there is jitter, you may need to modify some window size. Here are my versions

uname -a

Linux erwincoumans-Z170N-Gaming-5 5.4.0-53-generic #59~18.04.1-Ubuntu SMP Wed Oct 21 12:14:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

pip3 list
erwincoumans@erwincoumans-Z170N-Gaming-5:~/develop/motion_imitation$ pip3 list
Package                 Version             Location                                                            
----------------------- ------------------- --------------------------------------------------------------------
absl-py                 0.10.0              
alabaster               0.7.8               
appdirs                 1.4.3               
apturl                  0.5.2               
asn1crypto              0.24.0              
astor                   0.7.1               
attrs                   20.2.0              
audioread               2.1.6               
Babel                   2.4.0               
backcall                0.2.0               
baselines               0.1.5               
Brlapi                  0.6.6               
CacheControl            0.12.6              
certifi                 2019.11.28          
cffi                    1.12.3              
cfgv                    3.2.0               
chardet                 3.0.4               
Click                   7.0                 
cloudpickle             1.3.0               
colorama                0.4.3               
command-not-found       0.3                 
contextlib2             0.6.0               
cryptography            2.1.4               
cupshelpers             1.0                 
cycler                  0.10.0              
Cython                  0.28.2              
dataclasses             0.7                 
decorator               4.4.2               
defer                   1.0.6               
dill                    0.2.9               
distlib                 0.3.1               
distro                  1.4.0               
distro-info             0.18ubuntu0.18.04.1 
docutils                0.14                
filelock                3.0.12              
filterpy                1.4.5               
future                  0.18.2              
gast                    0.2.2               
git-review              1.26.0              
google-pasta            0.2.0               
grpcio                  1.20.0              
gym                     0.17.2              
h5py                    2.9.0               
html5lib                1.0.1               
httplib2                0.9.2               
identify                1.5.2               
idna                    2.8                 
imagesize               0.7.1               
importlab               0.5.1               
importlib-metadata      1.7.0               
importlib-resources     3.0.0               
inputs                  0.5                 
ipaddr                  2.2.0               
ipython                 7.16.1              
ipython-genutils        0.2.0               
iso8601                 0.1.12              
jedi                    0.17.2              
Jinja2                  2.10                
joblib                  0.13.2              
Keras                   2.2.4               
Keras-Applications      1.0.8               
Keras-Preprocessing     1.0.9               
keyring                 10.6.0              
keyrings.alt            3.0                 
kiwisolver              1.2.0               
language-selector       0.1                 
launchpadlib            1.10.6              
lazr.restfulclient      0.13.5              
lazr.uri                1.0.3               
lcm                     1.4.0               /home/erwincoumans/develop/locomotion_simulation2/src/lcm/lcm-python
librosa                 0.6.3               
llvmlite                0.28.0              
lockfile                0.12.2              
locomotion-simulation   0.0.1               /home/erwincoumans/develop/locomotion_simulation                    
louis                   3.5.0               
macaroonbakery          1.1.3               
Mako                    1.0.7               
Markdown                3.1                 
MarkupSafe              1.0                 
matplotlib              3.3.2               
meshcat                 0.0.19              
mock                    2.0.0               
mpi4py                  3.0.3               
msgpack                 0.6.2               
netifaces               0.10.4              
networkx                2.5                 
ninja                   1.10.0.post2        
nodeenv                 1.5.0               
numba                   0.43.1              
numpngw                 0.0.8               
numpy                   1.19.2              
oauth                   1.0.1               
olefile                 0.45.1              
opencv-python           4.1.0.25            
opt-einsum              3.3.0               
packaging               20.3                
pandas                  1.1.3               
parso                   0.7.1               
pbr                     5.1.3               
pep517                  0.8.2               
pexpect                 4.2.1               
pickleshare             0.7.5               
Pillow                  8.0.0               
pip                     19.0.3              
pre-commit              2.7.1               
progress                1.5                 
progressbar2            3.39.3              
prompt-toolkit          3.0.8               
protobuf                3.7.1               
pybullet                3.0.6               
pybullet-physx          2.4.3               /home/erwincoumans/develop/pybullet_physx/examples/pybullet/gym     
pycairo                 1.16.2              
pycapnp                 0.6.4               
pycparser               2.19                
pycrypto                2.6.1               
pycups                  1.9.73              
pydynamixel             1.0.0               
pyglet                  1.5.0               
Pygments                2.2.0               
pygobject               3.26.1              
pymacaroons             0.13.0              
PyNaCl                  1.1.2               
pyngrok                 5.0.0               
pyparsing               2.4.6               
pyRFC3339               1.0                 
python-apt              1.6.5+ubuntu0.2     
python-dateutil         2.8.1               
python-debian           0.1.32              
python-utils            2.3.0               
pytoml                  0.1.21              
pytype                  2020.9.16           
pytz                    2018.3              
pyxdg                   0.25                
PyYAML                  5.3.1               
pyzmq                   19.0.2              
quadprog                0.1.7               
reportlab               3.4.0               
requests                2.22.0              
requests-unixsocket     0.1.5               
resampy                 0.2.1               
retrying                1.3.3               
roman                   2.0.0               
scikit-learn            0.20.3              
scipy                   1.5.2               
screen-resolution-extra 0.0.0               
SecretStorage           2.3.1               
setuptools              41.0.0              
simplejson              3.13.2              
six                     1.14.0              
SoundFile               0.10.2              
Sphinx                  1.6.7               
sphinx-rtd-theme        0.2.4               
ssh-import-id           5.7                 
stable-baselines        2.10.0              
system-service          0.3                 
systemd-python          234                 
tensorboard             1.15.0              
tensorflow              1.15.4              
tensorflow-estimator    1.15.1              
tensorflow-gpu          1.13.1              
termcolor               1.1.0               
testresources           2.0.1               
toml                    0.10.1              
torch                   1.7.0               
tornado                 6.1                 
tqdm                    4.49.0              
traitlets               4.3.3               
typed-ast               1.4.1               
typing                  3.7.4.1             
typing-extensions       3.7.4.3             
u-msgpack-python        2.7.1               
ubuntu-drivers-common   0.0.0               
ufw                     0.36                
unattended-upgrades     0.1                 
urllib3                 1.25.8              
usb-creator             0.3.3               
virtualenv              20.0.31             
wadllib                 1.3.2               
wcwidth                 0.2.5               
webencodings            0.5.1               
Werkzeug                0.15.2              
wheel                   0.33.1              
wrapt                   1.12.1              
xkit                    0.0.0               
zipp                    3.1.0               
zope.interface          4.3.2  

[a2821952]

I'm wondering which computer on A1 you are using for MPC, Tx2 in the front part? or Upboard in the back?

[erwincoumans]

I'm using a fast AAM Ryzen or Intel computer, connected over ethernet to either port.

Did you resolve the issue?

Can you update to latest?