How do I run evaluate.py

[whenflyfish]

Sorry to bother. I don't have carla-testing/Town03. How can I get this file.

[MariaPriisalu]

Hi,

You should create a CARLA test set. I have not been able to figure out the best way to release the dataset yet (It is 100GB when zipped), but I am looking into this. For now you can recreate the datasets with the provided scripts.

RL/evaluate.py asks for data in the path self.carla_path_test as defined in RL/settings.py (make sure that self.new_carla=False in settings.py), (see RL/RLmain line 1352). Use the settings.py as specified in localData/Models depending on the model you choose to use. I would suggest setting USE_LOCAL_PATHS=1. Please set self.real_time_carla=False and self.real_time_carla_only=False.

The original CARLA dataset has been gathered with CARLA 0.8.2. You can find the client script for gathering the data in CARLA_simulation_client/gather_dataset.py. Basically we are gathering 3D reconstructions of scenes (i.e. depth+images+ semantic segmentation) every 50th frame and images and bounding boxes of moving objects at every frame. Gather a training set (path to folder in settings.py self.carla_path) with the script on CARLA_simulation_client/gather_dataset.py Town 1 and all of the test data is created on Town 2 , which should be placed in self.carla_path_test. You can change output_folder as necessary.

Next to combine the static objects in the .ply (viewable in Meshlab) files run carla_utils/combine_reconstructions.py

After this you should be able to run RL/evaluate.py.

Hope this helps, Maria Priisalu

On Wed, Dec 8, 2021 at 6:56 AM whenflyfish @.***> wrote:

Sorry to bother. I don't have carla-testing/Town03. How can I get this file.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MariaPriisalu/spl/issues/2, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKAU36D75RISNWZIQ3YPIR3UP3XRZANCNFSM5JS6RPLA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[whenflyfish]

Thank you very much for your reply. I only look at CARLA_0.9.4. I thought the CARLA_0.9.4 and CARLA_0.8.2  were the same :).I'm truly grateful for your help.

------------------ 原始邮件 ------------------ 发件人: "MariaPriisalu/spl" @.>; 发送时间: 2021年12月9日(星期四) 凌晨0:52 @.>; @.**@.>; 主题: Re: [MariaPriisalu/spl] How do I run evaluate.py (Issue #2)

Hi,

You should create a CARLA test set. I have not been able to figure out the best way to release the dataset yet (It is 100GB when zipped), but I am looking into this. For now you can recreate the datasets with the provided scripts.

RL/evaluate.py asks for data in the path self.carla_path_test as defined in RL/settings.py (make sure that self.new_carla=False in settings.py), (see RL/RLmain line 1352). Use the settings.py as specified in localData/Models depending on the model you choose to use. I would suggest setting USE_LOCAL_PATHS=1. Please set self.real_time_carla=False and self.real_time_carla_only=False.

The original CARLA dataset has been gathered with CARLA 0.8.2. You can find the client script for gathering the data in CARLA_simulation_client/gather_dataset.py. Basically we are gathering 3D reconstructions of scenes (i.e. depth+images+ semantic segmentation) every 50th frame and images and bounding boxes of moving objects at every frame. Gather a training set (path to folder in settings.py self.carla_path) with the script on CARLA_simulation_client/gather_dataset.py Town 1 and all of the test data is created on Town 2 , which should be placed in self.carla_path_test. You can change output_folder as necessary.

Next to combine the static objects in the .ply (viewable in Meshlab) files run carla_utils/combine_reconstructions.py

After this you should be able to run RL/evaluate.py.

Hope this helps, Maria Priisalu

On Wed, Dec 8, 2021 at 6:56 AM whenflyfish @.***> wrote:

> Sorry to bother. > I don't have carla-testing/Town03. > How can I get this file. > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <https://github.com/MariaPriisalu/spl/issues/2&gt;, or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AKAU36D75RISNWZIQ3YPIR3UP3XRZANCNFSM5JS6RPLA&gt; > . > Triage notifications on the go with GitHub Mobile for iOS > <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&amp;mt=8&amp;pt=524675&gt; > or Android > <https://play.google.com/store/apps/details?id=com.github.android&amp;referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub&gt;. > >

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[whenflyfish]

How to fix settings.py, for example self.useRLToyCar、self.useHeroCar.

[MariaPriisalu]

Just use the settings file that is in the folder with the network weights. See https://github.com/MariaPriisalu/spl/tree/master/localData/Models.

Best, Maria

On Mon, Dec 13, 2021 at 10:29 AM whenflyfish @.***> wrote:

How to fix settings.py, for example self.useRLToyCar、self.useHeroCar.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MariaPriisalu/spl/issues/2#issuecomment-992271080, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKAU36CXLLKCU7YTXSJF72DUQW4I3ANCNFSM5JS6RPLA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[whenflyfish]

Sorry to bother. I set car=CarAgent(settings, car_net, None) and agent = GoalAgent(settings),but episode.init_cars = [].

[MariaPriisalu]

Hi,

You shouldn't change anything in the code. Change settings by changing the values of the variables in setting.py instead.

Best, Maria

On Tue, Dec 14, 2021 at 10:18 AM whenflyfish @.***> wrote:

Sorry to bother. I set car=CarAgent(settings, car_net, None) and agent = GoalAgent(settings),but episode.init_cars = [].

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MariaPriisalu/spl/issues/2#issuecomment-993334597, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKAU36GKOP25STICYPQZBRDUQ4DU7ANCNFSM5JS6RPLA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[whenflyfish]

If I don't change the code. There will be a LocalError 'car' referenced before assigment.

------------------ 原始邮件 ------------------ 发件人: "MariaPriisalu/spl" @.>; 发送时间: 2021年12月14日(星期二) 下午5:25 @.>; @.**@.>; 主题: Re: [MariaPriisalu/spl] How do I run evaluate.py (Issue #2)

Hi,

You shouldn't change anything in the code. Change settings by changing the values of the variables in setting.py instead.

Best, Maria

On Tue, Dec 14, 2021 at 10:18 AM whenflyfish @.***> wrote:

> Sorry to bother. I set car=CarAgent(settings, car_net, None) and agent = > GoalAgent(settings),but episode.init_cars = []. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <https://github.com/MariaPriisalu/spl/issues/2#issuecomment-993334597&gt;, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AKAU36GKOP25STICYPQZBRDUQ4DU7ANCNFSM5JS6RPLA&gt; > . > Triage notifications on the go with GitHub Mobile for iOS > <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&amp;mt=8&amp;pt=524675&gt; > or Android > <https://play.google.com/store/apps/details?id=com.github.android&amp;referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub&gt;. > >

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[MariaPriisalu]

Hi,

What model are you trying to run? It looks like you are using incorrect settings (I suspect that you have self.goal_agent=True but you should have self.goal_agent= False). Just replace RL/settings.py with the content of one of the following settings files.

If you are trying to run the goal reaching carla pedestrian then please use the settings file here: https://github.com/MariaPriisalu/spl/blob/master/localData/Models/SemanticPedestrian/carla-goal-reaching/settings.py for goal free carla agent use https://github.com/MariaPriisalu/spl/blob/master/localData/Models/SemanticPedestrian/carla-goal-free/settings.py

For the adversarial test synthesizer use the following. For the occluded model use: https://github.com/MariaPriisalu/spl/blob/master/localData/Models/BehaviorVariedTestCaseGeneration/BaseModelOccludedwithCollisionAvoidingPedestrian/settings.py For the unoccluded model use: https://github.com/MariaPriisalu/spl/blob/master/localData/Models/BehaviorVariedTestCaseGeneration/BaseModelwithCollisionAvoidingPedestrian/settings.py

Best, Maria

On Tue, Dec 14, 2021 at 11:18 AM whenflyfish @.***> wrote:

If I don't change the code. There will be a LocalError 'car' referenced before assigment.

------------------ 原始邮件 ------------------ 发件人: "MariaPriisalu/spl" @.>; 发送时间: 2021年12月14日(星期二) 下午5:25 @.>; @.**@.>; 主题: Re: [MariaPriisalu/spl] How do I run evaluate.py (Issue #2)

Hi,

You shouldn't change anything in the code. Change settings by changing the values of the variables in setting.py instead.

Best, Maria

On Tue, Dec 14, 2021 at 10:18 AM whenflyfish @.***> wrote:

> Sorry to bother. I set car=CarAgent(settings, car_net, None) and agent = > GoalAgent(settings),but episode.init_cars = []. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > < https://github.com/MariaPriisalu/spl/issues/2#issuecomment-993334597&gt;, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AKAU36GKOP25STICYPQZBRDUQ4DU7ANCNFSM5JS6RPLA&gt;

> . > Triage notifications on the go with GitHub Mobile for iOS > < https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&amp;mt=8&amp;pt=524675&gt;

> or Android > < https://play.google.com/store/apps/details?id=com.github.android&amp;referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub&gt;.

> >

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MariaPriisalu/spl/issues/2#issuecomment-993388092, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKAU36GW2AOZJV5F45YAS7TUQ4KXBANCNFSM5JS6RPLA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[whenflyfish]

I use this setting.py https://github.com/MariaPriisalu/spl/blob/master/localData/Models/BehaviorVariedTestCaseGeneration/BaseModelOccludedwithCollisionAvoidingPedestrian/settings.py This will set self.random_agent=False、set self.goal_agent= False、self.social_lstm=False.The following if will be executed. if not settings.random_agent and not settings.goal_agent and not settings.social_lstm: config = tf.ConfigProto() config.gpu_options.allow_growth = True sess=tf.Session(graph=tf.get_default_graph(), config=config) agent and car is not initialized properly.

[MariaPriisalu]

This settings file and code run as they are. Your issue lies somewhere else, most likely in the paths. Please check that the path for loading the network-weights (see load_net_weights in RL/RLmain.py) is correct (you will need to change the path or move the weights). You might have to set USE_LOCAL_PATHS=1 in RL/settings.py.

On Tue, Dec 14, 2021 at 12:47 PM whenflyfish @.***> wrote:

I use this setting.py

https://github.com/MariaPriisalu/spl/blob/master/localData/Models/BehaviorVariedTestCaseGeneration/BaseModelOccludedwithCollisionAvoidingPedestrian/settings.py This will set self.random_agent=False、set self.goal_agent= False、self.social_lstm=False.The following if will be executed. if not settings.random_agent and not settings.goal_agent and not settings.social_lstm: config = tf.ConfigProto() config.gpu_options.allow_growth = True sess=tf.Session(graph=tf.get_default_graph(), config=config) agent and car is not initialized properly.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MariaPriisalu/spl/issues/2#issuecomment-993460665, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKAU36GGOXYZV2FQO67ABRTUQ4VEBANCNFSM5JS6RPLA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[whenflyfish]

Thanks for your answers. But load_net_weights never get called before call self.evaluate_method_carla(agent, sess, settings, viz, car=car)(RLmain.py,1231). I don't have poses_hips.mat. Is it necessary? And I really want to understand that when I call self.evaluate_method_carla, does car have to != None. If car !=None, where to initialize the car in the code.

[MariaPriisalu]

Hi,

Hi, thanks for writing back. I have had to remove some parts of the code due to licensing so an if statement became an elif. Now everything will hopefully work.

You don't need poses_hips.mat.

Now load_net_weights is called on line 1206. See lines 1210-1215 in RL/RLmain.py these set the pedestrian and the car agent. Lines 1192-1203 set a baseline agent. So ignore these. The agent is then called for the first time on line 1231 in self. evaluate_method_carla(agent, sess, settings, viz, car=car).

Best, Maria

On Wed, Dec 15, 2021 at 10:20 AM whenflyfish @.***> wrote:

Thanks for your answers. But load_net_weights never get called before call self.evaluate_method_carla(agent, sess, settings, viz, car=car)(RLmain.py,1231). I don't have poses_hips.mat. Is it necessary? And I really want to understand that when I call self.evaluate_method_carla, does car have to != None. If car !=None, where to initialize the car in the code.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MariaPriisalu/spl/issues/2#issuecomment-994561317, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKAU36EJTJSRAM546BCOF4TURBMVFANCNFSM5JS6RPLA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[whenflyfish]

Sorry to bother. If evaluation.py runs properly, what files does it generate. I generate the gent_pos_4_test0-64_0.npy、init_car.npy、init_stat_map.npy、learn_car.npy、poses.npy and an empty file agent_pos_0 folder . And how do I visualize those files.

[MariaPriisalu]

Hi,

Your filenames should include a timestamp for example 2021-10-27-21-14-35.842985. To visualize results insert the timestamp of the run into RL/visualization_scripts/Visualize_evaluation.py

timestamps=[ "2021-10-27-21-14-35.842985"]

and run the file with

python RL/visualization_scripts/Visualize_evaluation.py

from spl/.

Best,

Maria

On Tue, Dec 21, 2021 at 9:54 AM whenflyfish @.***> wrote:

Sorry to bother. If evaluation.py runs properly, what files does it generate. I generate the gent_pos_4_test0-64_0.npy、init_car.npy、init_stat_map.npy、learn_car.npy、poses.npy and an empty file agent_pos_0 folder . And how do I visualize those files.

— Reply to this email directly, view it on GitHub https://github.com/MariaPriisalu/spl/issues/2#issuecomment-998588299, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKAU36HXNB2CLVIL4QJHYPTUSA6EBANCNFSM5JS6RPLA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>