search

sbft-cps-tool-competition / cps-tool-competition

The repository hosts the code for the SBFT CPS Tool competition for testing autonomous cars.
https://sbft24.github.io/tools/
Other
15 stars 16 forks source link

Pipeline is getting stuck #3

Open cetinkaya opened 1 year ago

cetinkaya commented 1 year ago

We identified some tests (attached two examples) for which the code_pipeline (py.exe competition.py ...) gets stuck.

We noticed that the runtime gets stuck in beamngpy(version 1.24)/connection.py's _recv_exactly function (line 295) while waiting for some data from a TCP socket (connected to BeamNG). BeamNG process seems to exit after running the roads in the example files. A TCP connection is made to BeamNG, but nothing is received from TCP socket before it exits. We were suspecting it may have something to do with having short roads.

The example cases below can be run as standard test generators.

Case 1

from code_pipeline.tests_generation import RoadTestFactory

class TGenerator:
    def __init__(self, executor=None, map_size=None):
        self.executor = executor
        self.map_size = map_size

    def start(self):
        while True:
            if self.executor.is_over():
                return

            road_points = [(10, 10), (11.344603355928557, 10.060079259050896), (12.6847426705547, 10.184956476892312),
                           (14.017297083470007, 10.374340844680106), (15.339163397668056, 10.627791332016203),
                           (16.647263306139877, 10.944717714001753), (17.938550560506627, 11.324381945728193),
                           (19.210018064995438, 11.76589988100538), (20.470583754304556, 12.237643291218049),
                           (21.719529220583045, 12.739343325506312), (22.956142678450778, 13.270714060457768),
                           (24.17971937065137, 13.83145266305786), (25.38956196969975, 14.421239563277163),
                           (26.58498097529546, 15.039738636197253), (27.76529510727511, 15.686597393571349),
                           (28.92983169388018, 16.361447184710578), (30.07792705511873, 17.063903406581343),
                           (31.20892688100271, 17.793565722994114), (32.322186604445136, 18.55001829275863),
                           (33.41707176860474, 19.332830006675607), (34.49295838846872, 20.141554733229768),
                           (35.54923330646744, 20.975731572844264), (36.67419854045589, 21.714663859633625),
                           (37.85929071162123, 22.352726746804695), (39.095488747237624, 22.885063218296725),
                           (40.373382550222246, 23.30762106100699), (41.68324462999411, 23.617183710753654),
                           (43.00013810572321, 23.895319065796055), (44.32330906107626, 24.141867894737807),
                           (45.651999985887564, 24.356689049344432), (46.985450209830645, 24.539659545350172),
                           (48.322896337898996, 24.690674632865992), (49.663572687446724, 24.809647856348484),
                           (51.006711726538846, 24.89651110409533), (52.3515445133603, 24.951214647238974),
                           (53.69730113643213, 24.9737271682162)]

            the_test = RoadTestFactory.create_road_test(road_points)
            test_outcome, description, execution_data = self.executor.execute_test(the_test)

Case 2

from code_pipeline.tests_generation import RoadTestFactory

class TGenerator:
    def __init__(self, executor=None, map_size=None):
        self.executor = executor
        self.map_size = map_size

    def start(self):
        while True:
            if self.executor.is_over():
                return

            road_points = [(10, 35.901957044342275), (10.20558638939485, 34.57180594817386), (10.495193593412237, 33.25738768539611),
             (10.867653471641402, 31.96400400633875), (11.321463694619853, 30.696871817629663),
             (11.854793803529844, 29.461102139626313), (12.465492593415668, 28.261679490943028),
             (13.15109679014128, 27.10344178322726), (13.908840986089501, 25.99106080728007),
             (14.647838574157202, 24.86613846908398), (15.367882030415629, 23.728990667238566),
             (16.141353190979256, 22.627487119391898), (16.966490822405042, 21.56413600553181),
             (17.766959958396853, 20.48209309304314), (18.542337328107422, 19.381930542924675),
             (19.29221292867533, 18.26423009744952), (20.016190242025708, 17.129582772552304),
             (20.71388644454045, 15.978588545312675), (21.384932609487088, 14.811856036700334),
             (22.02897390209934, 13.630002189749376), (22.645669767206066, 12.433651943332087),
             (23.234694109309515, 11.223437901704724), (23.79573546501762, 10.0)]

            the_test = RoadTestFactory.create_road_test(road_points)
            test_outcome, description, execution_data = self.executor.execute_test(the_test)
cetinkaya commented 1 year ago

Waiting a little bit (with time.sleep) after the "taskkill" operation here seems to solve the problem.