search

openai / mujoco-py

MuJoCo is a physics engine for detailed, efficient rigid body simulations with contacts. mujoco-py allows using MuJoCo from Python 3.
Other
2.83k stars 810 forks source link

Can't render the model which include deformable objects #373

Open hyc6668378 opened 5 years ago

hyc6668378 commented 5 years ago

Describe the bug The .xml file which I used include cloth (composite). If I render the model,like 

viewer = MjViewer(sim)
viewer.render()

It will come out the bug below

Expected behavior I really want to use the Deformable objects in simulation. and I want to observe what happened by render

Error Messages

Creating window glfw
Segmentation fault (core dumped)

Desktop:

Environment

evelyd commented 5 years ago

I am having the exact same issue. With the deformable object, I get a segmentation fault. With the same .xml and the deformable object commented out, I do not get the error.

I have the same Desktop profile as above.

Has this bug been identified yet?

fhln commented 5 years ago

this same error: Segmentation fault. Please help.

rojikada commented 5 years ago

Same problem here. Both in virtualbox Ubuntu 16.04, python3.7, mujoco2 and also in macos 10.14.4, python3.7, mujoco2. From /proc/cpuinfo:

processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 70
model name  : Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
stepping    : 1
cpu MHz     : 2494.226
cache size  : 6144 KB
physical id : 0
siblings    : 1
core id     : 0
cpu cores   : 1
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 13
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq monitor ssse3 cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase avx2 invpcid flush_l1d
bugs        : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds
bogomips    : 4988.45
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

GDB output from sim.render(600, 600):

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff5241a1d in XQueryExtension ()
   from /usr/lib/x86_64-linux-gnu/libX11.so.6
(gdb) backtrace
#0  0x00007ffff5241a1d in XQueryExtension ()
   from /usr/lib/x86_64-linux-gnu/libX11.so.6
#1  0x00007ffff5235562 in XInitExtension ()
   from /usr/lib/x86_64-linux-gnu/libX11.so.6
#2  0x00007ffff67c4afc in ?? () from /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1
#3  0x00007ffff67c0241 in glXQueryVersion ()
   from /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1
#4  0x00007ffff7ba0b34 in glxewInit ()
   from /usr/lib/x86_64-linux-gnu/libGLEW.so
#5  0x00007fffed0d9adf in mjr_makeContext ()
   from /home/X/Software/mujoco200_linux/bin/libmujoco200.so
#6  0x00007fffed81b2b1 in __pyx_pf_9mujoco_py_4cymj_15MjRenderContext_6_set_mujoco_buffers (__pyx_v_self=0x1208798)
    at /tmp/pip-install-3exlshgv/mujoco-py/mujoco_py/cymj.c:140096
#7  __pyx_pw_9mujoco_py_4cymj_15MjRenderContext_7_set_mujoco_buffers (
    __pyx_v_self=0x1208798, unused=<optimized out>)
    at /tmp/pip-install-3exlshgv/mujoco-py/mujoco_py/cymj.c:9003
#8  0x00007fffed7892f1 in __Pyx_PyObject_CallMethO (arg=0x0, 
    func=0x7fffeaffa750)
    at /tmp/pip-install-3exlshgv/mujoco-py/mujoco_py/cymj.c:196756
#9  __Pyx_PyObject_CallNoArg (func=0x7fffeaffa750)
    at /tmp/pip-install-3exlshgv/mujoco-py/mujoco_py/cymj.c:174
#10 0x00007fffed81b08b in __pyx_pf_9mujoco_py_4cymj_15MjRenderContext_2__init__
    (__pyx_v_opengl_backend=<optimized out>, 
    __pyx_v_device_id=<optimized out>, __pyx_v_offscreen=<optimized out>, 
    __pyx_v_sim=<optimized out>, __pyx_v_self=<optimized out>)
    at /tmp/pip-install-3exlshgv/mujoco-py/mujoco_py/cymj.c:139784
#11 __pyx_pw_9mujoco_py_4cymj_15MjRenderContext_3__init__ (
    __pyx_v_self=<optimized out>, __pyx_args=<optimized out>, 
    __pyx_kwds=<optimized out>)
--Type <RET> for more, q to quit, c to continue without paging--c
    at /tmp/pip-install-3exlshgv/mujoco-py/mujoco_py/cymj.c:8349
#12 0x0000000000529c19 in ?? ()
#13 0x00007fffed6d8e30 in __Pyx_PyObject_Call (func=0x1073278, arg=0x7ffff7f50630, kw=0x7fffeb012168) at /tmp/pip-install-3exlshgv/mujoco-py/mujoco_py/cymj.c:196434
#14 0x00007fffed7d79ec in __pyx_pf_9mujoco_py_4cymj_5MjSim_10render (__pyx_v_device_id=<optimized out>, __pyx_v_mode=<optimized out>, __pyx_v_depth=<optimized out>, __pyx_v_camera_name=<optimized out>, __pyx_v_height=<optimized out>, __pyx_v_width=<optimized out>, __pyx_v_self=0x7fffeaf91840) at /tmp/pip-install-3exlshgv/mujoco-py/mujoco_py/cymj.c:131842
#15 __pyx_pw_9mujoco_py_4cymj_5MjSim_11render (__pyx_v_self=0x7fffeaf91840, __pyx_args=<optimized out>, __pyx_kwds=<optimized out>) at /tmp/pip-install-3exlshgv/mujoco-py/mujoco_py/cymj.c:588
#16 0x00000000004e8873 in _PyMethodDef_RawFastCallKeywords ()
#17 0x00000000005e67e9 in _PyMethodDescr_FastCallKeywords ()
#18 0x000000000055f90c in _PyEval_EvalFrameDefault ()
#19 0x000000000055a140 in _PyEval_EvalCodeWithName ()
#20 0x0000000000559ec3 in PyEval_EvalCode ()
#21 0x000000000062a7f2 in ?? ()
#22 0x000000000062ac5a in PyRun_FileExFlags ()
#23 0x000000000062aa17 in PyRun_SimpleFileExFlags ()
#24 0x00000000006040b5 in ?? ()
#25 0x0000000000603d3a in _Py_UnixMain ()
#26 0x00007ffff6a3a830 in __libc_start_main (main=0x4e4f30 <main>, argc=2, argv=0x7fffffffdb68, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdb58) at ../csu/libc-start.c:291
#27 0x0000000000603c29 in _start ()

Onscreen/offscreen rendering does not matter. Same problem even when using python 3.6.5_1. Problem seems somewhere within the rendering:

import mujoco_py as mjp

model = mjp.load_model_from_path("model/mjmodel2.xml")
sim = mjp.MjSim(model)

# to speed up computation we need the off screen rendering
viewer = mjp.MjRenderContextOffscreen(sim, 0)
viewer.render(420, 380, 0) # crashes here

(gdb) run test.py 
Starting program: /usr/bin/python3.7 test.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[Detaching after fork from child process 9577]
[Detaching after fork from child process 9578]
[Detaching after fork from child process 9579]
[Detaching after fork from child process 9580]
[Detaching after fork from child process 9586]
[Detaching after fork from child process 9592]
[Detaching after fork from child process 9598]
Creating window glfw

Program received signal SIGSEGV, Segmentation fault.
__memset_avx2 () at ../sysdeps/x86_64/multiarch/memset-avx2.S:135
135 ../sysdeps/x86_64/multiarch/memset-avx2.S: Adresář nebo soubor neexistuje.
(gdb) backtrace
#0  __memset_avx2 () at ../sysdeps/x86_64/multiarch/memset-avx2.S:135
#1  0x00007fffed092131 in mjv_updateSkin ()
   from /home/X/Software/mujoco200_linux/bin/libmujoco200.so
#2  0x00007fffed829639 in __pyx_pf_9mujoco_py_4cymj_15MjRenderContext_14render
    (__pyx_v_camera_id=<optimized out>, __pyx_v_height=<optimized out>, 
    __pyx_v_width=<optimized out>, __pyx_v_self=0x16d8258)
    at /tmp/pip-install-3exlshgv/mujoco-py/mujoco_py/cymj.c:141530
#3  __pyx_pw_9mujoco_py_4cymj_15MjRenderContext_15render (
    __pyx_v_self=0x16d8258, __pyx_args=<optimized out>, 
    __pyx_kwds=<optimized out>)
    at /tmp/pip-install-3exlshgv/mujoco-py/mujoco_py/cymj.c:10002
#4  0x00000000004e8873 in _PyMethodDef_RawFastCallKeywords ()
#5  0x00000000005e67e9 in _PyMethodDescr_FastCallKeywords ()
#6  0x000000000055f90c in _PyEval_EvalFrameDefault ()
#7  0x000000000055a140 in _PyEval_EvalCodeWithName ()
#8  0x0000000000559ec3 in PyEval_EvalCode ()
#9  0x000000000062a7f2 in ?? ()
#10 0x000000000062ac5a in PyRun_FileExFlags ()
#11 0x000000000062aa17 in PyRun_SimpleFileExFlags ()
#12 0x00000000006040b5 in ?? ()
#13 0x0000000000603d3a in _Py_UnixMain ()
#14 0x00007ffff6a3a830 in __libc_start_main (main=0x4e4f30 <main>, argc=2, 
    argv=0x7fffffffdb68, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fffffffdb58)
    at ../csu/libc-start.c:291
#15 0x0000000000603c29 in _start ()
rojikada commented 5 years ago

Ok, found where the problem is, its the mjv_updateScene(self._model_ptr, self._data_ptr, &self._vopt, &self._pert, &self._cam, mjCAT_ALL, &self._scn), made fork: https://github.com/wutter/mujoco-py/blob/master/mujoco_py/mjrendercontext.pyx pip3 install --user --force-reinstall --no-compile git+https://github.com/wutter/mujoco-py/tree/5ba1983931925b27e9b2e542c1a709b3ac106891

if you try file from previous post, you can see it falls on update:

Creating window glfw
Init render
Will set buffer size
Will update scene
Segmentation fault: 11

With that line commented out, it gets to render and fall there (as there is nothing to render I guess).

Next might be checking the differences between vars for soft-body scene vs non-soft body one and vars from basic.cpp from mujoco installation for the soft-body model (because it works there), but I do not know how to do that.

rojikada commented 5 years ago

Until solved, going with tempomary solution: visual/viewer in C++ connected via UDP/file IO to python. Qpos are than being send as timeframes and viewer loads them.

rojikada commented 5 years ago

Found out where the problem is. Its not deformable object itself, but the skin. It seems like skins are not supported for some reason. If you remove skin, even the carpet works.

yilin-wu98 commented 5 years ago

I encounter the same bug and after removing the skin part, the render works, but I need the skin attribute to make the structure look like a cloth. I am just wondering if anyone knows in which way I can use the skin because without the skin, I can only see the internal structure of a cloth composite object, lacking the visual attribute.

usut commented 5 years ago

I have the same problem. It works without skin. But I need skin. How to add skin in mujoco-py? Really appreciate it if anyone helps.

usut commented 5 years ago

Found out where the problem is. Its not deformable object itself, but the skin. It seems like skins are not supported for some reason. If you remove skin, even the carpet works.

Hello, have you solved this problem? Have you made it work with skin? Thanks.

rojikada commented 5 years ago

I did not bother to solve in python, working solution if you need the viewer is https://github.com/openai/mujoco-py/issues/373#issuecomment-494696192 I can post the code public (https://github.com/wutter/mujoco-py-to-c-viewer)

usut commented 5 years ago

I did not bother to solve in python, working solution if you need the viewer is #373 (comment) I can post the code public (will update this comment with git link)

It would be helpful if you could post the code with a link. Thanks :)

usut commented 5 years ago

I got it. Thank you very much.

coffeiersama commented 5 years ago

I did not bother to solve in python, working solution if you need the viewer is #373 (comment) I can post the code public (https://github.com/wutter/mujoco-py-to-c-viewer)

hi！ Thanks for your codes! I replace basic.cpp in my mujoco200 with your viewer.cpp and then make the file, then I try to run my project with skin, but there is still the same error, Is there any problem in my method of using your codes??

rojikada commented 5 years ago

Sorry, I did not put much docs on how to use.

  1. Modify your model (copy), so it does not use skin.
  2. Use your python code with the model without skin to generate the sim.data.qpos (and than export it to csv by file.writerow(sim.data.qpos).
  3. Use the csv with qpos + model with skin as inputs for the viewer (.cpp).

There is also space for improvement, it does export active forces, so you could display them, but I did not bother as it was not importatnt for my use.

coffeiersama commented 5 years ago

Sorry, I did not put much docs on how to use.

1. Modify your model (copy), so it does not use skin.

2. Use your python code with the model without skin to generate the sim.data.qpos (and than export it to csv by file.writerow(sim.data.qpos).

3. Use the csv with qpos + model with skin as inputs for the viewer (.cpp).

There is also space for improvement, it does export active forces, so you could display them, but I did not bother as it was not importatnt for my use.

Thanks for the quick reply, I have followed the instruction and get to see the cloth skin in a single frame by running the viewer, I was wondering whether i can use this to show a continuous animation by updating the csv file?

rojikada commented 5 years ago

Actually, it is already continuous in mujoco sense, that it generates 1 frame each timestep (setted up in mujoco)

rojikada commented 5 years ago

443#issuecomment-521965723 (just so its all at one place). Ah I see, like what do you mean continuous. That I did not solve. Most portable solution would probably be to use socket, but easier solution might be to have a lock file (random file) and the output, so the C-viewer would pre-load generated data when non-locked and the python would just generate some frame blocks (assuming the generator is faster than realtime) and while doing so, create the lock file.

coffeiersama commented 5 years ago

443#issuecomment-521965723 (just so its all at one place). Ah I see, like what do you mean continuous. That I did not solve. Most portable solution would probably be to use socket, but easier solution might be to have a lock file (random file) and the output, so the C-viewer would pre-load generated data when non-locked and the python would just generate some frame blocks (assuming the generator is faster than realtime) and while doing so, create the lock file.

Adam, thanks for your prompt response.

What I want to achieve is to detect the corner of a cloth when the cloth is being manipulated. We need the skin displayed all the time. It is infeasible to pause the motion and display skin as we do not know when to pause. Here, the cloth is being maniplated and move continuously.

Do you mean that this kind of continous motion has not been fixed?

padmaja-kulkarni commented 4 years ago

Has someone solved this issue? I want to create a GYM environment with a deformable object with the Fetch robot. I can include a rope but not a deformable object with Skin. Some comments talk about removing the skin. How does an object, e.g. a box would look like?

I am using this currently for loading a box (which does not work).

Thanks!

brinij commented 4 years ago

Is anyone working on solving this problem?

fantasyRqg commented 4 years ago

after this change. success to use model contain skin tag

jangirrishabh commented 4 years ago

Looking for solutions to this problem, the above mentioned solution does not work for me @fantasyRqg

fantasyRqg commented 4 years ago

Install instructions

pip3 uninstall  -y mujoco_py

git clone git@github.com:fantasyRqg/mujoco-py.git

cd mujoco-py

python3 setup.py build

python3 setup.py install

make sure you have installed the gcc Mac OS : brew install gcc Ubuntu: sudo apt-get install gcc

Here is softbox demo of mujoco image

@jangirrishabh

TMmichi commented 4 years ago

Install instructions

pip3 uninstall  -y mujoco_py

git clone git@github.com:fantasyRqg/mujoco-py.git

cd mujoco-py

python3 setup.py build

python3 setup.py install

make sure you have installed the gcc Mac OS : brew install gcc Ubuntu: sudo apt-get install gcc

Here is softbox demo of mujoco image

@jangirrishabh

Many thanks. It solved my problem. You are a genius