Open SrinivasuluCharupally opened 1 year ago
Yes, you need to run them parallely in 3 seperate terminals.
Thanks Mankaran. One more thing.. This is what i am getting when try to run . I am running from SSH.
`(flowpilot) che7@che7-w30871:~/flowpilot$ ./CarlaUE4.sh -RenderOffScreen 4.26.2-0+++UE4+Release-4.26 522 0 Disabling core dumps. WARNING: lavapipe is not a conformant vulkan implementation, testing use only.
che7@che7-w30871:~/flowpilot$ ./launchflowpilot.sh logmessaged thermald statsd keyvald flowpilot pandad deleter flowinitd logmessaged thermald statsd keyvald flowpilot pandad deleter flowinitd logmessaged thermald statsd keyvald flowpilot pandad deleter flowinitd logmessaged thermald_ statsd keyvald flowpilot pandad deleter flowinitd accepted_terms False completed_training False ignition False
che7@che7-w30871:~/flowpilot$ python3 tools/sim/bridge.py Restarting bridge. Error: time-out of 5000ms while waiting for the simulator, make sure the simulator is ready and connected to 127.0.0.1:2000 Restarting bridge. Error: time-out of 5000ms while waiting for the simulator, make sure the simulator is ready and connected to 127.0.0.1:2000 Restarting bridge. Error: time-out of 5000ms while waiting for the simulator, make sure the simulator is ready and connected to 127.0.0.1:2000
`
(flowpilot) che7@che7-w30871:~/flowpilot$ ./CarlaUE4.sh -RenderOffScreen
4.26.2-0+++UE4+Release-4.26 522 0
Disabling core dumps.
WARNING: lavapipe is not a conformant vulkan implementation, testing use only.
Is Carla running properly?
Not sure, doing it for the first time. i running it from SSH.
You will have to make sure carla is running. If you remove the RenderOffScreen
arg, it should show the carla window which will confirm its up and running
Could you please confirm this error. this is OK or, Not ?
(flowpilot) che7@che7-w30871:~/flowpilot$ python scripts/dummy_params.py
I think you compiled (ran scons
command) without entering the flowpilot env (pip env shell
)
Do pip env shell
then scons
in flowpilot root directory then this warning should go away
Got it, thanks Mankaran.
MankaranSingh - We tried the steps but that did not work for us. Attaching a screenRecording of what we did. Could tell us what was the issue ?
jFYI ... Prithiveen from our team tried the steps.
https://github.com/flowdriveai/flowpilot/assets/7656634/83bd1ae9-1baf-4457-a442-d073f2c09113
@SrinivasuluCharupally Can you share the output of following command.
file tmp.avi
@RaghavPrabhakar66
che7@che7-w30871:~/flowpilot/selfdrive/assets$ file tmp
tmp: RIFF (little-endian) data, AVI, 1164 x 874, 25.00 fps, video: Motion JPEG
@MankaranSingh @RaghavPrabhakar66 Able to successfully launch flowpilot app now :) bridge.py still renders a Segmentation fault (core dumped) error
https://github.com/flowdriveai/flowpilot/assets/99075098/f5949cfb-b580-4fb4-badc-8821c5c299f1
I'd recommed you clean the build using scons -c
.
I am not sure but maybe when you interchanged the python version (3.10) and ran scons that built some files that were not compatible.
@MankaranSingh , @RaghavPrabhakar66 - Could we have short call some time next week to look into the issues ? We are from Valeo team / org , trying to explore the option for different platform.
@MankaranSingh @RaghavPrabhakar66
Tired cleaning the build using scons -c
and then building again using scons
but bridge.py still renders a Segmentation fault (core dumped) error
user journey: https://drive.google.com/file/d/1h7DjBOh8l7pfuaAKOrPIbo6Kdaa1f4Sl/view?usp=sharing
@PrithiveenKumaarRamkumar @SrinivasuluCharupally I am not able to reproduce the issue. It would be helpful if you exactly mention what lines are causing segmenation fault in the script.
@SrinivasuluCharupally the issue is related to the carla-py package provided in this repo. you can try uninstalling that and do pip install carla==0.9.12
and rerun this.
@MankaranSingh - the segfault occurs when try to import the carla from bridge.py script
#pipenv shell
# scons
# python tools/sim/bridge.py
Segmentation fault
===> this seg fault occur from the line import carla
in the tools/sim/bridge.py
BTW - We tried the above steps with pip install carla==0.9.12
, pip3 install carla==0.9.12
and pip install -e carla-py
.
Result is same all the time, seg fault while importing carla into the bridge.py
.
I am sure some thing is missing in the README.
I also encountered the same problem. Have you solved the problem? If it is resolved, I would like to ask for advice on a solution
@MankaranSingh @RaghavPrabhakar66 Able to successfully launch flowpilot app now :) bridge.py still renders a Segmentation fault (core dumped) error flowpilot_app_journey.mp4
I also encountered the same problem. Have you solved the problem? If it is resolved, I would like to ask for advice on a solution
You can try installing carla like this https://github.com/carla-simulator/carla/issues/1466#issuecomment-522797557
@MankaranSingh - could you confirm these steps are correct for each terminal. We are running every thing from the pipenv shell
, check the below steps
pipenv shell
scons
./CarlaUE4.sh -RenderOffScreen
pipenv shell
launch_flowpilot.sh
pipenv shell
python tool/sim/bridge.py
Your steps are correct. you just need a working carla-python api. Please check this out https://github.com/carla-simulator/carla/issues/1466#issuecomment-522797557
你的步骤是正确的。 你只需要一个工作carla-蟒蛇。 请看看这个 卡拉-模拟器/卡拉#1466(注释)
This method won't work
@MankaranSingh In https://github.com/carla-simulator/carla/issues/1466#issuecomment-522797557 it is recommended to install carla by extracting the .egg files, but the .egg files are for python3.7.
However, activating the flowpilot environment pipenv shell
takes you to python3.9 environment
The Carla build documentation https://carla.readthedocs.io/en/latest/build_linux/ and discussions https://github.com/carla-simulator/carla/discussions/4519 says that the PythonAPI is supported till python3.8. I am now able to import carla successfully in python3.7 and python3.8
Using python3.8 I was able to generate the build using scons
and run the other scripts successfully but bridge.py renders the following traceback:
Traceback (most recent call last):
File "tools/sim/bridge.py", line 28, in <module>
pm = messaging.PubMaster(['roadCameraState', 'roadCameraBuffer', 'wideRoadCameraState',
File "/home/che7/flowpilot/cereal/messaging/__init__.py", line 257, in __init__
self.sock[s] = pub_sock(s)
File "/home/che7/flowpilot/cereal/messaging/__init__.py", line 47, in pub_sock
sock.connect(context, endpoint)
File "cereal/messaging/messaging_pyx.pyx", line 139, in cereal.messaging.messaging_pyx.PubSocket.connect
raise MultiplePublishersError
cereal.messaging.messaging_pyx.MultiplePublishersError
@PrithiveenKumaarRamkumar for MultiplePublishersError, you'd need to comment out the ROAD_CAMERA_SOURCE
and wide camera source. This will not start the video publishers and allow publihers from carla bridge script to take over
@MankaranSingh Now we are able to run Carla, launch the flowpilot app and run bridge.py from the python3.8 environment but we are not able to "take control of the wheel" in the flowpilot app. Are we missing something here? https://drive.google.com/file/d/1gLtdT1o6qeKijSyk_YwiPW62gROMnFJn/view?usp=sharing
https://github.com/flowdriveai/flowpilot/wiki/Running-On-Simulation
see the last step. The input keys would be taken in the bridge.py
terminal
@MankaranSingh - We are able to control the Carla output with 1, 2 and W-A-D keys. Could you tell us how to render with our own video with the environment, we have a local traffic video, would like to see the flowdriver output for that. Let us know the details.
@MankaranSingh - could you guide us on this ?
The video can be specified in ROAD_CAMERA_SOURCE
in launch script. By default, flowpilot desktop version run on a video only.
@MankaranSingh - do we have to launch all the three terminals with the "ROAD_CAMERA_SOURCE" enabled ? The reason is, we noticed following error with ROAD_CAMERA_SOURCE enabled , you suggested to disable it /comment out last time.
_Traceback (most recent call last):
File "tools/sim/bridge.py", line 28, in
ROAD_CAMERA_SOURCE needs to be disabled to accept stream from carla simulation. do not run carla or or bridge.py if you want to run on video
On Sun, Oct 1, 2023 at 2:06 PM Srinivasulu Charupally < @.***> wrote:
@MankaranSingh https://github.com/MankaranSingh - do we have to launch all the three terminals with the "ROAD_CAMERA_SOURCE" enabled ? The reason is, we noticed following error with ROAD_CAMERA_SOURCE enabled , you suggested to disable it /comment out.
Traceback (most recent call last): File "tools/sim/bridge.py", line 28, in pm = messaging.PubMaster(['roadCameraState', 'roadCameraBuffer', 'wideRoadCameraState',
— Reply to this email directly, view it on GitHub https://github.com/flowdriveai/flowpilot/issues/38#issuecomment-1742006470, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKHLNLGCXAN7HSIWLOSQWCLX5ETRHANCNFSM6AAAAAA2YQM3PE . You are receiving this because you were mentioned.Message ID: @.***>
Hi @MankaranSingh - Sorry for the late repose, we were busy with other projects. Still we are not able to render the video.
Attaching our "launch_flowpilot.sh" script.
pipenv shell scons ./launch_flowpilot.sh
attaching screen shot
Hi @MankaranSingh Do we have to make changes on these scripts for using user's input video? https://github.com/flowdriveai/flowpilot/blob/master/selfdrive/loggerd/video_process.py https://github.com/flowdriveai/flowpilot/blob/master/selfdrive/loggerd/config.py
The default launchfile when you download it freshly is taking in video input only. you do not need to run bridge.py.
after launch just run scripts/dummy_publishers.py
in seperate terminal
Thanks @MankaranSingh I am able to run it now. Attaching screen shots one more question : Would it display the bounding boxes on detected objects ?
?
This is an end to end driving model. this dosent outputs bounding boxes
@MankaranSingh - I have downloaded a video from web, mp4 format, and tried to render the video under flowDrive, but seeing following error. You have any idea ?
Video download link : https://www.pexels.com/video/cars-traveling-on-expressway-5382495/
convert video to avi format.
@MankaranSingh - I tried but did not work. Does it really requires Calibration & CAN signals ? it shows "CalibrationError" & "canError".
Hey @MankaranSingh
I followed the steps in here https://github.com/flowdriveai/flowpilot/wiki/Running-On-Simulation for running on desktop. I get flowpilot in infinite loop.
ubuntu 20.04
nvidia-driver-535(proprietary tested)
carla= 0.9.12
I've given ROAD_CAMERA_SOURCE="test.avi"
and Iaunched flowpilot following is the o/p I get on after running.
[main] INFO org.nd4j.nativeblas.NativeOpsHolder - Number of threads used for linear algebra: 2
[main] INFO org.nd4j.linalg.cpu.nativecpu.CpuNDArrayFactory - Binary level Generic x86 optimization level AVX/AVX2
[main] INFO org.nd4j.nativeblas.Nd4jBlas - Number of threads used for OpenMP BLAS: 2
[main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Backend used: [CPU]; OS: [Linux]
[main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Cores: [24]; Memory: [7.8GB];
[main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Blas vendor: [OPENBLAS]
[main] INFO org.nd4j.linalg.cpu.nativecpu.CpuBackend - Backend build information:
GCC: "7.5.0"
STD version: 201103L
DEFAULT_ENGINE: samediff::ENGINE_CPU
HAVE_FLATBUFFERS
HAVE_OPENBLAS
[ERROR:0] global /home/runner/work/opencv/opencv/opencv-4.5.1/modules/videoio/src/cap.cpp (142) open VIDEOIO(CV_IMAGES): raised OpenCV exception:
OpenCV(4.5.1) /home/runner/work/opencv/opencv/opencv-4.5.1/modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): /home/rp/Downloads/testhd.avi in function 'icvExtractPattern'
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by ai.flow.common.BufferUtils (file:/home/rp/flowpilot/common/build/libs/common-0.0.1.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of ai.flow.common.BufferUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[main] INFO ai.flow.launcher.FlowInitd - Connected to the FlowInit daemon
[ERROR:0] global /home/runner/work/opencv/opencv/opencv-4.5.1/modules/videoio/src/cap.cpp (142) open VIDEOIO(CV_IMAGES): raised OpenCV exception:
OpenCV(4.5.1) /home/runner/work/opencv/opencv/opencv-4.5.1/modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): /home/rp/flowpilot/selfdrive/assets/videos/intro.avi in function 'icvExtractPattern'
Exception in thread "camerad:roadCameraState" CvException [org.opencv.core.CvException: cv::Exception: OpenCV(4.5.1) /home/runner/work/opencv/opencv/opencv-4.5.1/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
]
at org.opencv.imgproc.Imgproc.cvtColor_1(Native Method)
at org.opencv.imgproc.Imgproc.cvtColor(Imgproc.java:6842)
at ai.flow.sensor.camera.CameraManager.processFrame(CameraManager.java:112)
at ai.flow.sensor.camera.CameraManager.run(CameraManager.java:165)
at java.base/java.lang.Thread.run(Thread.java:829)
Is there any hardcoded path like /home/runner/*
somewhere in flowpilot
If I comment out ROAD_CAMERA_SOURCE
and start carla, flowpilot is always into infinite loop and nothing is detected.
@piyushmz what is test.avi ? where are you placing it ?
so I was testing with a downloaded mp4 file which I converted to avi and I placed the path in ROAD_CAMERA_SOURCE
but it didn't work. The selfdrive/assets/tmp
does work with flowpilot. As of bridging of carla with flowpilot I get segmentation fault.
Hi, I am trying to setup it on PC / Desktop, Intel CPU + Ubuntu 20.04. I followed the install steps from here : https://github.com/flowdriveai/flowpilot/wiki/Installation#installing-on-desktop Then I followed "Running on Simulator" steps from here : https://github.com/flowdriveai/flowpilot/wiki/Running-On-Simulation#following-steps-will-help-you-get-carla-simulation---flowpilot-integration-up-and-running
I got confused with these following two steps... Could you help to clarigy
This means, total 3 terminals do i need to launch, one for "./CarlaUE4.sh -RenderOffScreen", one for "./launch_flowpilot.sh" and one for "python tool/sim/bridge.py"
Is that correct ? and, Do i need to have Camera or SmartPhone to run the application ?