lgsvl / simulator

A ROS/ROS2 Multi-robot Simulator for Autonomous Vehicles
Other
2.29k stars 780 forks source link

INTEGRATION: [PythonAPI + AutowareAI] or [VSE + AutowareAI] running in Docker (AutowareAI version: 1.14.0) Ubuntu 20.04 #1341

Closed marcusvinicius178 closed 3 years ago

marcusvinicius178 commented 3 years ago

Hi Colleagues. 2 brief questions

1) Does PythonAPI works along AutowareAI navigation stack on SVL? 2) Does Virtual Scene Editor (VSE) Simulator works along AutowareAI on SVL?

I tried both of them, following the documentation steps, and afterwards tried to run the documentation demo (https://www.svlsimulator.com/docs/system-under-test/autoware-instructions/) over the customized scenarion (both with Python API scenario and later with a VSE built scenario). The result is that the car did not move to the goal spot ( set by rviz nav_goal icon). If I am not wrong the car localized itself using the lidar localization pkg (with rviz pose_estimation icon) and also switched the lights on. But did not move.

A) An output of PythonAPI - AutowareAI integration is below

Integration_Python_Autoware Python_API_Autoware_Integration

I got some errors (in terminal in picture above). But seems the integration iPython_ Autoware is possible?

B) The attempt of VSE and AutowareAI integration also failed and the output is below:

integration_AI_VSE Integration_VSE_AutowareAI_failed

I do not understand why the integration failed. If I am launching the same demo packages that worked before )without the VSE or Python integration). Can someone assist me here? Is there a source code working, demo or videotutorial of VSE or PythonAPI + AutowareAI integration that you could provide me please?

At last but not less important the other attempts to lauch VSE based simulation ALWAYS FAIL and return me "cluster busy" errors, and the simulation open and closes if few seconds. Why does this happen? How to fix it?

Thanks in advance.

EricBoiseLGSVL commented 3 years ago

@marcusvinicius178 It seems you have many errors and issues so let's try to break this down to solve each one.

1) What hardware are you using? 2) What simulator version? API version? and Autoware.AI version? 3) What type of simulator are you running? Editor? Binary? Have you edited the source code at all? 4) Thank you for the images but we need logs. Please find them here 5) Since it seems you are having issues with the WebUI, we will need the error from the browser with F12 and please screen shot the error in the webUI as well.

I am adding other developers on the team here so we can figure out the issue and help others that might be having the same problem. @lemketron @hadiTab @eagafonovlge @cyberphysicalrobotics

Please refrain from posting in other issues, we will resolve this issue here.

marcusvinicius178 commented 3 years ago

Hi @EricBoiseLGSVL thanks very much to answer me so quickly!!! I will refrain to ask more similar questions. I just with to set up a custom scenario with VSE (preferrable) or PythonAPI and afterwards test AutowareAI path planning module to perform evasive maneuvers from the created agents by means of VSE/PythonAPi.

Answering your questions:

1- What hardware are you using?

A)My system features are below (sorry it is in portuguese but easy to infer) Sistema

B)Nvidia/Cuda sistema_nvidia

2) What simulator version? API version? and Autoware.AI version? A) Simulator Version: 2021.1 simulator B) API Version: Is there more than one version? I do not know I just dowloaded from here: https://github.com/lgsvl/PythonAPI Python_API Version C) AutowareAI version: 1.14.0 Autoware_AI_Version

3) What type of simulator are you running? Editor? Binary? Have you edited the source code at all? A) The simulator is 2021.1 simulator B) Binary, dowloaded from the SVL website after signup for linux machines C) No I did not edit the simulator source code.

4) Thank you for the images but we need logs. Please find them here I just found 2 logs in .config/unity3d/LGEletronics/SVLSimulator folder: Player Log and Player-prev.log. They are attached: Player-prev.log Player.log

Logs The upm log and Editor.log are not in the folder your link "here" suggested: Empty_log

5) Since it seems you are having issues with the WebUI, we will need the error from the browser with F12 and please screen shot the error in the webUI as well.

Ok I will send the Error with F12 pressed for the 2 cases ( A-PythonAPI+AutowareAI and B-VSE + AutowareAI autopilot) I am trying to run and let the simulator opened (Without close after 10 seconds as it is happening)

A)PythonAPI + AutowareAI autopilot bridge API_Autopilot API_Autoware WhatsApp Image 2021-04-07 at 21 24 15 (1) API_Autoware_terminal WhatsApp Image 2021-04-07 at 21 24 15

B) Visual Scene Editor (VE) + AutowareAI autopilot bridge F12 F12_VSE VSE_F12

The attempt to integrate AutowareAI with PythonAPI was almost done successfully. After several attempts I recognized the possible issues:

A) Autopilot Icon on Quick Start tab must be activated? B) The desired Mission Planning and Motion Planning must be selected on Computing Tab in AutowareAI? However this does not make too sense, because the Mission_Planning and Motion_Planning modules loads already the launch demo launch files that contain the planners to be used C) The TF or/and Vehicle model icons on Setup tab must be activated? d) The Point Cloud Icon on MAp tab must be swiched-on? The localization in rviz is not working, just rarely works and show where the car is. e) The icon LGSVL Simulator must be selected on Simulation Tab? It is other simulator release right? Anyway clicking over it the simulator is never opened...just Gazebo simulator opens when clicking over it.

I could not recognize specifically the reason why the car is not located in rviz (or in very rare attempts it is located, usually after trigger the nav_goal icon and when this last icon works, which is also not often...) and why when the car accepts a goal spot on rviz, the car does not start moving on simulator.

Well the pictures are not so good because m computer has 2 screens (so it is kind of small) but I have done a video of the attempts of the desired integration (A Python API+ AutowareAI , B- VSE+AutowareAI) and it is in link below:

A) Python+ AutowareAI https://www.youtube.com/watch?v=KgB-t0tMxXk

B) VSE+ AutowareAI https://www.youtube.com/watch?v=crb7QoS8N3E obs: When accidentaly I let the runtime-manager switched on with Autopilot and all other modules on ( on beginning of this video) the simulation last more time to close (you can see that the pedestrian and npc vehicle moves a lot). Then I closed the runtime-manager and just played the simulator. The result is that the simulation closes too quickly before the agents move. Is there a relationship of the Autopilot be activated in the first time?

Using The PythonApi script on the video I did not create agents. But the only explanation for the simulator be opened infinitely and not close is that I added the code line sim.run(0) To let the simulation running. Otherwise the other python scripts have the same behaviour that VSE had, open and close the simulation after some seconds? What is the reason?

I am adding other developers on the team here so we can figure out the issue and help others that might be having the same problem. @lemketron @hadiTab @eagafonovlge @cyberphysicalrobotics

Ok I hope we fing together a solution.

Thanks very much for the support!

Please refrain from posting in other issues, we will resolve this issue here.

Ok I will!

EricBoiseLGSVL commented 3 years ago

Excellent, thanks for all the details, this helps immensely. One thing worries me. You are using a 1050ti with only 4gb of memory. Are you running autoware and svl sim on the same machine?

marcusvinicius178 commented 3 years ago

Hi @EricBoiseLGSVL yes unfortunately I have just this notebook. I also don't know how to use 2 notebooks and communicate them ( one for the SVL simulator and other for the AutowareAI?) Should I use other notebook is it because of this the simulator open and close? I am not expert in PC. Do you know if is it possible to buy and add memory to my notebook in addition to this 4Gb? Or is it not possible (it is a fixed slot)? Just say me what I need to buy for the computer store and I will try to increase this memory. Which memory type and how much? Also another components if necessary ( I have 1 TB of ssd cause guys said ssd performance is better than Hd, therefore I changed) I will be very grateful if you suggested me possibilities to work around this in the easier way $. I mean without need to buy another computer...is Possible to rent online computers to run the simulation? Thanks in advance

zerolover commented 3 years ago

In your video, the localization module of Autoware may not work properly since there is not Point Cloud shown in Rviz. Check the documentation demo carefully, and you should try build-in Simulation first.

marcusvinicius178 commented 3 years ago

@zerolover built in is = built from source you mean, but why? I followed the documentation they also used docker image. I accessed the same release 1.14.0 with $ ./sh - t 1.14.0

The lidar pc worked in localization module, when I use these same demo launch files integrated with a random scenario simulation in interactive mode with AutowareAi 1.14 autopilot. All the modules worked properly as in documentation. However when launching these demos along with Python API simulation, then the same localization module that worked previously does not work. But now I recognized I used a different car (used Jaguar for PythonApi and Lexus for The demo documentation, maybe this be the reason why the point cloud did not work) I will try again with Lexus now.

EricBoiseLGSVL commented 3 years ago

@marcusvinicius178 I have covered the specs questions here #1352 I am going to answer the questions for this issue and mark it close soon. It is too large to be helpful to the community and with too many off topic questions buried in the text/images.

As for the questions at the top:

1) Yes, Python API works with simulator while AutowareAI is running 2) Yes, VSE works with Autoware AI and Simulator. VSE creates JSON files that are converted to Python API code and run in the Runner.

As you get your setup working, please open issues that you find and keep them concise so others that have the same issue can find and be helped.

As for the logs, it says that the ros bridge has failed because connection is refused. Do you have any anti virus running? If so disable and also make sure the bridge is setup correctly. Look at the linked issue for ROS bridge links.

zerolover commented 3 years ago

@marcusvinicius178 Hmmm, I run the Autoware with python API successfully. I watch your video again and find the problem. The main reason is there is no bridge between SVL and Autoware. Check your bridge is connected, see https://www.svlsimulator.com/docs/user-interface/simulation-menu/#bridge-menu or use "rostopic echo /points_raw --noarr" to see whether Autoware receives the topic. What's your code(python script) for running the python API. You should try code under https://github.com/lgsvl/PythonAPI/tree/master/quickstart first, especially 22-connecting-bridge.py!

zerolover commented 3 years ago

This is my code

#!/usr/bin/env python3
#
# Copyright (c) 2019-2021 LG Electronics, Inc.
#
# This software contains code licensed as described in LICENSE.
#

import time
from environs import Env
import lgsvl

env = Env()

sim = lgsvl.Simulator("127.0.0.1", 8181)
if sim.current_scene == lgsvl.wise.DefaultAssets.map_borregasave:
    sim.reset()
else:
    sim.load(lgsvl.wise.DefaultAssets.map_borregasave)

spawns = sim.get_spawn()

state = lgsvl.AgentState()
state.transform = spawns[0]
forward = lgsvl.utils.transform_to_forward(spawns[0])
right = lgsvl.utils.transform_to_right(spawns[0])

# add EGO
state.transform.position = spawns[0].position - (0.0 * right)
ego = sim.add_agent("fbeb6e0f-6020-4996-87a5-d56f907ba1ed", lgsvl.AgentType.EGO, state)

# add obstacle
state.transform.position = spawns[0].position + (75 * forward) - (0.2 * right)
state.transform.rotation = spawns[0].rotation
layer_mask = 0
for bit in [0, 10, 11, 12]: # do not put 9 here, to not hit EGO vehicle itself
  layer_mask |= 1 << bit
hit = sim.raycast(state.transform.position, lgsvl.Vector(0, -1, 0), layer_mask)
state.transform.position = hit.point
sim.add_agent("Sedan", lgsvl.AgentType.NPC, state)
spawns = sim.get_spawn()

# The EGO is now looking for a bridge at the specified IP and port
ego.connect_bridge(env.str("LGSVL__AUTOPILOT_0_HOST", "127.0.0.1"), env.int("LGSVL__AUTOPILOT_0_PORT", 9090))

sim.run()
EricBoiseLGSVL commented 3 years ago

Thanks for the help @zerolover :)

marcusvinicius178 commented 3 years ago

Hi Thanks I was a beginner with SVL weeks ago. You were right @zerolover The issues why the simulator closes were always the following ones:

1 - Forget to launch the bridge properly with Autoware.AI ( I mean it is necessary befor play the simulation to Launch the "SENSING" tab on Autoware.AI runtime manager interface)

2 - Incorrect Python API code.

However lastly I had also another unexpected close behaviour in simulator while trying to use maps from Apollo 5.0 into Apollo 6.0: https://github.com/ApolloAuto/apollo/issues/13774 If you know something about please let me know