search

xbpeng / DeepMimic

Motion imitation with deep reinforcement learning.
https://xbpeng.github.io/projects/DeepMimic/index.html
MIT License
2.29k stars 484 forks source link

Error in `python': corrupted size vs. prev_size #52

Open bsivanantham opened 5 years ago

bsivanantham commented 5 years ago

hi, @xbpeng Please help me with this error . I tried to reinstall complete NVIDIA driver and Bullet but still no change . I am stuck with this error. python3.6 mpi_run.py --arg_file args/train_humanoid3d_run_args.txt --num_workers 4 Running with 4 workers cmd: mpiexec -n 4 python DeepMimic_Optimizer.py --arg_file args/train_humanoid3d_run_args.txt --num_workers 4 Successfully to load args from: args/train_humanoid3d_run_args.txt scene imitate

Error in python: corrupted size vs. prev_size: 0x0000563c02ad4060 ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fa2b12e47e5] /lib/x86_64-linux-gnu/libc.so.6(+0x7e9dc)[0x7fa2b12eb9dc] /lib/x86_64-linux-gnu/libc.so.6(+0x81cde)[0x7fa2b12eecde] /lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7fa2b12f1184] /usr/lib/x86_64-linux-gnu/libLinearMath.so.2.83(+0x36d3)[0x7fa28b9716d3] /usr/lib/x86_64-linux-gnu/libBulletDynamics.so.2.83(_ZN11btMultiBodyC1EifRK9btVector3bbb+0xe93)[0x7fa28bf1a9b3] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN10cMultiBodyC1EifRK9btVector3bbb+0x14)[0x7fa28cda9614] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN13cSimCharacter14BuildMultiBodyERSt10shared_ptrI10cMultiBodyE+0xa3)[0x7fa28cdafb33] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN13cSimCharacter12BuildSimBodyERKNS_7tParamsE+0x130)[0x7fa28cdaf430] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN13cSimCharacter4InitERKSt10shared_ptrI6cWorldERKNS_7tParamsE+0x118)[0x7fa28cdaaa68] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN13cSceneSimChar15BuildCharactersEv+0x19f)[0x7fa28ce190ff] /home/deepmimic/DeepMimic/DeepMimicCore

======= Memory map: ======== [none:30562] Process received signal [none:30562] Signal: Aborted (6) [none:30562] Signal code: (-6) [none:30562] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f36a273c390] [none:30562] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7f36a2396428] [none:30562] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7f36a239802a] [none:30562] [ 3] /lib/x86_64-linux-gnu/libc.so.6(+0x777ea)[0x7f36a23d87ea] [none:30562] [ 4] /lib/x86_64-linux-gnu/libc.so.6(+0x7e9dc)[0x7f36a23df9dc] [none:30562] [ 5] /lib/x86_64-linux-gnu/libc.so.6(+0x81cde)[0x7f36a23e2cde] [none:30562] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_malloc+0x54)[0x7f36a23e5184] [none:30562] [ 7] /usr/lib/x86_64-linux-gnu/libLinearMath.so.2.83(+0x36d3)[0x7f367ca9a6d3] [none:30562] [ 8] /usr/lib/x86_64-linux-gnu/libBulletDynamics.so.2.83(_ZN11btMultiBodyC1EifRK9btVector3bbb+0xe93)[0x7f367d0439b3] [none:30562] [ 9] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN10cMultiBodyC1EifRK9btVector3bbb+0x14)[0x7f367ded2614] [none:30562] [10] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN13cSimCharacter14BuildMultiBodyERSt10shared_ptrI10cMultiBodyE+0xa3)[0x7f367ded8b33] [none:30562] [11] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN13cSimCharacter12BuildSimBodyERKNS_7tParamsE+0x130)[0x7f367ded8430] [none:30562] [12] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN13cSimCharacter4InitERKSt10shared_ptrI6cWorldERKNS_7tParamsE+0x118)[0x7f367ded3a68] [none:30562] [13] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN13cSceneSimChar15BuildCharactersEv+0x19f)[0x7f367df420ff] [none:30562] [14] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZTv0_n304_N13cSceneImitate15BuildCharactersEv+0x34)[0x7f367df3b6d4] [none:30562] [15] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN13cSceneSimChar4InitEv+0x4c)[0x7f367df415dc] [none:30562] [16] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN15cRLSceneSimChar4InitEv+0x25)[0x7f367df3dc05] [none:30562] [17] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZTv0_n40_N13cSceneImitate4InitEv+0x95)[0x7f367df3b225] [none:30562] [18] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(_ZN14cDeepMimicCore10SetupSceneEv+0x268)[0x7f367de19948] [none:30562] [19] /home/deepmimic/DeepMimic/DeepMimicCore/_DeepMimicCore.so(+0x1f6d9a)[0x7f367df63d9a] [none:30562] [20] python(_PyCFunction_FastCallDict+0x91)[0x5575a3148681] [none:30562] [21] python(+0x19842c)[0x5575a31cf42c] [none:30562] [22] python(_PyEval_EvalFrameDefault+0x30a)[0x5575a31f438a] [none:30562] [23] python(+0x19253b)[0x5575a31c953b] [none:30562] [24] python(+0x198505)[0x5575a31cf505] [none:30562] [25] python(_PyEval_EvalFrameDefault+0x30a)[0x5575a31f438a] [none:30562] [26] python(+0x191a76)[0x5575a31c8a76] [none:30562] [27] python(_PyFunction_FastCallDict+0x1bc)[0x5575a31c9c4c] [none:30562] [28] python(_PyObject_FastCallDict+0x26f)[0x5575a3148b0f] [none:30562] [29] python(_PyObject_Call_Prepend+0x63)[0x5575a314d6a3] [none:30562] End of error message

mpiexec noticed that process rank 1 with PID 30560 on node none exited on signal 6 (Aborted).

Process finished with exit code 0

error.txt

Complete error file has been attached for your reference . Looking forward for your reply

Thanks

xbpeng commented 5 years ago

sorry, not sure what might be causing this. Does running: DeepMimic.py --arg_file args/train_humanoid3d_run_args.txt work?

bsivanantham commented 5 years ago

@xbpeng running DeepMimic.py --arg_file args/train_humanoid3d_run_args.txt

Successfully to load args from: args/train_humanoid3d_run_args.txt Renderer: GeForce GTX 970/PCIe/SSE2 OpenGL version supported 4.6.0 NVIDIA 390.87 Compiling shader: data/shaders/Mesh_VS.glsl Compiling shader: data/shaders/VertColor_PS.glsl Compiling shader: data/shaders/FullScreenQuad_VS.glsl Compiling shader: data/shaders/DownSample_PS.glsl Compiling shader: data/shaders/Mesh_VS.glsl Compiling shader: data/shaders/DownSample_PS.glsl scene imitate [none:17055] Process received signal [none:17055] Signal: Bus error (7) [none:17055] Signal code: (128) [none:17055] Failing at address: (nil)

i am getting above error.

xbpeng commented 5 years ago

It looks like it might be a graphics driver issue. Have you tried updating it?

bsivanantham commented 5 years ago

@xbpeng even I thought the same and tried to reinstall the GPU drivers .. Do you know any specific version which I should have ??

From the previous issue posted I thought NVIDIA 390.87 was suitable version. If you have any other configuration which I can try please let me know.