bark-simulator / bark

Open-Source Framework for Development, Simulation and Benchmarking of Behavior Planning Algorithms for Autonomous Driving
https://bark-simulator.github.io/
MIT License
287 stars 69 forks source link

strange phenomenon #518

Closed tinmodeHuang closed 3 years ago

tinmodeHuang commented 3 years ago

info printed as below:

I0410 13:30:18.583123 140075456821056 sac_runner.py:94] Training iterations 23000-24000 took 3026.243 seconds (avg. 3.026s / iteration).
I0410 13:30:26.928425 140075456821056 tfa_runner.py:123] The ego agent reached its goal. 
I0410 13:31:02.116097 140075456821056 tfa_runner.py:123] The ego agent reached its goal. 
I0410 13:31:08.224915 140075456821056 tfa_runner.py:123] The ego agent reached its goal. 
I0410 13:31:21.354181 140075456821056 tfa_runner.py:123] The ego agent reached its goal. 
I0410 13:31:41.352142 140075456821056 tfa_runner.py:123] The ego agent reached its goal. 
I0410 13:31:48.012688 140075456821056 tfa_runner.py:148] The agent achieved an average reward of -0.600, collision-rate of 0.72000, took on average 41.720 steps, and reached the goal  0.200 (evaluated over 25 episodes).
I0410 13:31:48.049430 140075456821056 common.py:1003] Saved checkpoint: /home/tinmode/bark-ml/logs/episode_duration:14s,replaybuffer:100_000,Tau:0.05,interval:3/ckpt-24001
I0410 13:31:48.049718 140075456821056 tfa_agent.py:84] Saved checkpoint for step 24001.
 0# bark::commons::SegfaultHandler(int) in /home/tinmode/.cache/bazel/_bazel_tinmode/0534ffbce5f0177dc35c2eb30e06c204/execroot/bark_ml/bazel-out/k8-fastbuild/bin/examples/tfa.runfiles/bark_project/bark/core.so
 1# 0x00007F65DB7D0040 in /lib/x86_64-linux-gnu/libc.so.6
 2# 0x00007F652FAC0DE4 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
 3# 0x00007F652FAC1FBB in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
 4# 0x00007F652FAC2096 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
 5# 0x00007F652FAC23FD in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
 6# 0x00007F652FAC2953 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
 7# 0x00007F652FAC2FA4 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
 8# 0x00007F652FAC3799 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
 9# 0x00007F652FA51CD8 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
10# 0x00007F652FAC3DEC in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
11# 0x00007F652FAC4325 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
12# 0x00007F652FAC4832 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
13# 0x00007F652FAC4EC5 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
14# 0x00007F652FAC510E in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
15# 0x00007F652FA509E6 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
16# 0x00007F652FA57391 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
17# 0x00007F652FA3F37A in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
18# 0x00007F652FA43205 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/ft2font.cpython-37m-x86_64-linux-gnu.so
19# _PyMethodDef_RawFastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
20# 0x000000000054A9C0 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
21# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
22# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
23# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
24# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
25# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
26# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
27# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
28# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
29# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
30# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
31# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
32# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
33# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
34# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
35# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
36# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
37# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
38# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
39# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
40# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
41# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
42# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
43# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
44# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
45# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
46# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
47# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
48# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
49# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
50# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
51# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
52# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
53# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
54# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
55# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
56# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
57# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
58# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
59# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
60# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
61# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
62# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
63# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
64# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
65# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
66# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
67# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
68# 0x00000000004D8102 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
69# PyObject_Call in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
70# 0x00007F652CFEB58D in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/lib-dynload/_tkinter.cpython-37m-x86_64-linux-gnu.so
71# TclInvokeStringCommand in /usr/lib/x86_64-linux-gnu/libtcl8.6.so
72# TclNRRunCallbacks in /usr/lib/x86_64-linux-gnu/libtcl8.6.so
73# 0x00007F652C69506F in /usr/lib/x86_64-linux-gnu/libtcl8.6.so
74# TclServiceIdle in /usr/lib/x86_64-linux-gnu/libtcl8.6.so
75# Tcl_DoOneEvent in /usr/lib/x86_64-linux-gnu/libtcl8.6.so
76# 0x00007F652C95EDEC in /usr/lib/x86_64-linux-gnu/libtk8.6.so
77# TclNRRunCallbacks in /usr/lib/x86_64-linux-gnu/libtcl8.6.so
78# 0x00007F652CFED4F0 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/lib-dynload/_tkinter.cpython-37m-x86_64-linux-gnu.so
79# _PyMethodDescr_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
80# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
81# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
82# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
83# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
84# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
85# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
86# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
87# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
88# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
89# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
90# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
91# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
92# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
93# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
94# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
95# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
96# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
97# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
98# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
99# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
100# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
101# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
102# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
103# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
104# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
105# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
106# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
107# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
108# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
109# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
110# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
111# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
112# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
113# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
114# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
115# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
116# 0x00000000004D8102 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
117# PyObject_Call in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
118# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
119# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
120# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
121# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
122# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
123# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
124# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
125# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
126# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
127# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
128# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
129# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
130# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
131# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
132# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
133# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
134# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
135# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
136# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
137# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
138# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
139# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
140# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
141# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
142# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
143# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
144# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
145# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
146# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
147# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
148# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
149# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
150# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
151# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
152# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
153# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
154# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
155# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
156# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
157# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
158# _PyFunction_FastCallDict in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
159# 0x00000000004D8102 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
160# PyObject_Call in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
161# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
162# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
163# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
164# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
165# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
166# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
167# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
168# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
169# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
170# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
171# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
172# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
173# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
174# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
175# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
176# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
177# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
178# _PyFunction_FastCallKeywords in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
179# 0x000000000054A880 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
180# _PyEval_EvalFrameDefault in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
181# _PyEval_EvalCodeWithName in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
182# PyEval_EvalCode in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
183# 0x00000000006308E2 in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
184# PyRun_FileExFlags in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
185# PyRun_SimpleFileExFlags in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
186# 0x000000000065450E in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
187# _Py_UnixMain in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3
188# __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6
189# _start in /home/tinmode/bark-ml/bark_ml/python_wrapper/venv/bin/python3

terminate called without an active exception
Fatal Python error: Aborted

Thread 0x00007f6271f5d700 (most recent call first):
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 470 in _handle_results
  File "/usr/lib/python3.7/threading.py", line 870 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007f627275e700 (most recent call first):
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 422 in _handle_tasks
  File "/usr/lib/python3.7/threading.py", line 870 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007f6272f5f700 (most recent call first):
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 413 in _handle_workers
  File "/usr/lib/python3.7/threading.py", line 870 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007f6384ff9700 (most recent call first):
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 110 in worker
  File "/usr/lib/python3.7/threading.py", line 870 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007f64a77fe700 (most recent call first):
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 470 in _handle_results
  File "/usr/lib/python3.7/threading.py", line 870 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007f64a7fff700 (most recent call first):
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 422 in _handle_tasks
  File "/usr/lib/python3.7/threading.py", line 870 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007f64aca08700 (most recent call first):
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 413 in _handle_workers
  File "/usr/lib/python3.7/threading.py", line 870 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Thread 0x00007f64ad209700 (most recent call first):
  File "/usr/lib/python3.7/multiprocessing/pool.py", line 110 in worker
  File "/usr/lib/python3.7/threading.py", line 870 in run
  File "/usr/lib/python3.7/threading.py", line 926 in _bootstrap_inner
  File "/usr/lib/python3.7/threading.py", line 890 in _bootstrap

Current thread 0x00007f65dbd87740 (most recent call first):
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/backends/backend_agg.py", line 203 in draw_text
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/text.py", line 729 in draw
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/artist.py", line 41 in draw_wrapper
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/image.py", line 131 in _draw_list_compositing_images
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/axes/_base.py", line 2747 in draw
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/cbook/deprecation.py", line 411 in wrapper
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/artist.py", line 41 in draw_wrapper
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/image.py", line 131 in _draw_list_compositing_images
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/figure.py", line 1864 in draw
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/artist.py", line 41 in draw_wrapper
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/backends/backend_agg.py", line 407 in draw
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/backends/backend_tkagg.py", line 9 in draw
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/backends/_backend_tk.py", line 253 in idle_draw
  File "/usr/lib/python3.7/tkinter/__init__.py", line 749 in callit
  File "/usr/lib/python3.7/tkinter/__init__.py", line 1705 in __call__
  File "/usr/lib/python3.7/tkinter/__init__.py", line 1177 in update
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/backends/_backend_tk.py", line 390 in flush_events
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/backend_bases.py", line 2407 in start_event_loop
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/matplotlib/pyplot.py", line 438 in pause
  File "/home/tinmode/.cache/bazel/_bazel_tinmode/0534ffbce5f0177dc35c2eb30e06c204/execroot/bark_ml/bazel-out/k8-fastbuild/bin/examples/tfa.runfiles/bark_project/bark/runtime/viewer/matplotlib_viewer.py", line 157 in show
  File "/home/tinmode/.cache/bazel/_bazel_tinmode/0534ffbce5f0177dc35c2eb30e06c204/execroot/bark_ml/bazel-out/k8-fastbuild/bin/examples/tfa.runfiles/bark_project/bark/runtime/viewer/matplotlib_viewer.py", line 145 in drawWorld
  File "/home/tinmode/.cache/bazel/_bazel_tinmode/0534ffbce5f0177dc35c2eb30e06c204/execroot/bark_ml/bazel-out/k8-fastbuild/bin/examples/tfa.runfiles/bark_project/bark/runtime/runtime.py", line 69 in render
  File "/home/tinmode/.cache/bazel/_bazel_tinmode/0534ffbce5f0177dc35c2eb30e06c204/execroot/bark_ml/bazel-out/k8-fastbuild/bin/examples/tfa.runfiles/bark_ml/bark_ml/environments/single_agent_runtime.py", line 61 in step
  File "/home/tinmode/.cache/bazel/_bazel_tinmode/0534ffbce5f0177dc35c2eb30e06c204/execroot/bark_ml/bazel-out/k8-fastbuild/bin/examples/tfa.runfiles/bark_ml/bark_ml/library_wrappers/lib_tf_agents/tfa_wrapper.py", line 67 in _step
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/environments/py_environment.py", line 203 in step
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/environments/batched_py_environment.py", line 166 in _step
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/environments/py_environment.py", line 203 in step
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/environments/tf_py_environment.py", line 314 in _step_py
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/environments/tf_py_environment.py", line 214 in _execute
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/environments/tf_py_environment.py", line 318 in _isolated_step_py
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tensorflow/python/ops/script_ops.py", line 573 in py_func_common
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tensorflow/python/ops/script_ops.py", line 686 in numpy_function
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py", line 201 in wrapper
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/environments/tf_py_environment.py", line 335 in _step
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py", line 620 in wrapper
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/environments/tf_environment.py", line 241 in step
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/drivers/dynamic_episode_driver.py", line 131 in loop_body
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2735 in while_loop
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2499 in while_loop_v2
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 605 in new_func
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/drivers/dynamic_episode_driver.py", line 224 in _run
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/utils/common.py", line 185 in with_check_resource_vars
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/tf_agents/drivers/dynamic_episode_driver.py", line 195 in run
  File "/home/tinmode/.cache/bazel/_bazel_tinmode/0534ffbce5f0177dc35c2eb30e06c204/execroot/bark_ml/bazel-out/k8-fastbuild/bin/examples/tfa.runfiles/bark_ml/bark_ml/library_wrappers/lib_tf_agents/runners/sac_runner.py", line 54 in _train
  File "/home/tinmode/.cache/bazel/_bazel_tinmode/0534ffbce5f0177dc35c2eb30e06c204/execroot/bark_ml/bazel-out/k8-fastbuild/bin/examples/tfa.runfiles/bark_ml/bark_ml/library_wrappers/lib_tf_agents/runners/tfa_runner.py", line 87 in Train
  File "/home/tinmode/.cache/bazel/_bazel_tinmode/0534ffbce5f0177dc35c2eb30e06c204/execroot/bark_ml/bazel-out/k8-fastbuild/bin/examples/tfa.runfiles/bark_ml/examples/tfa.py", line 89 in run_configuration
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/absl/app.py", line 251 in _run_main
  File "/home/tinmode/bark-ml/bark_ml/python_wrapper/venv/lib/python3.7/site-packages/absl/app.py", line 303 in run
  File "/home/tinmode/.cache/bazel/_bazel_tinmode/0534ffbce5f0177dc35c2eb30e06c204/execroot/bark_ml/bazel-out/k8-fastbuild/bin/examples/tfa.runfiles/bark_ml/examples/tfa.py", line 100 in <module>

thanks for any tips at advance

patrickhart commented 3 years ago

Hello there,

what code base is that based on? The current master and in which cases does this error occur? It seems like this is a segmentation fault, in which case there is most probably an access violation (e.g., on an object that has been deleted).

Best,

Patrick

EDIT: what kind of code are you executing?

tinmodeHuang commented 3 years ago

the snippet below to determine the id of lane with agent in: agent.cpp

XodrLaneId Agent::GetCurrentLaneId() const {
  auto lanecorridor = road_corridor_->GetCurrentLaneCorridor(GetCurrentPosition());
  auto lane = lanecorridor->GetCurrentLane(GetCurrentPosition());
  return lane->GetId();
}

agent.cpp

.def("GetDistance2RoadEdge", &Agent::GetDistance2RoadEdge)

I doubt that the function causes the problem when running tfa_gnn.py

patrickhart commented 3 years ago

What do you use the lane_id for? At first sight, this function seems to be correct.

I would recommend you to assert that the values are not null pointers, e.g., using BARK_EXPECT_TRUE(lanecorridor != nullptr);

Do you access anything with the retrieved lane_id? But in general debugging and setting assertions should help narrowing down the problem.

tinmodeHuang commented 3 years ago

the answer to the first question is to determine Ids of lane with each agent in.

patrickhart commented 3 years ago

The function looks good on first sight, but as I cannot see the full code and how you are using it, you have to try to debug it by yourself. You can also attach the python process to the c++.

Have a look here: https://bark-simulator.readthedocs.io/en/latest/debugging.html#debugging-c-and-python

tinmodeHuang commented 3 years ago

I tried to debug it but was blocked by the problem below, and it is a custom function

image

patrickhart commented 3 years ago

As I do not have access to any of your code, I cannot help you further at this point. I would suggest for you to add as many assertions as you can to figure out where exactly it fails.

tinmodeHuang commented 3 years ago

all right