Get ros_gmsl_driver working properly

[ZileiChen]

A 3rd party source: https://github.com/BeckerFelix/ros_gmsl_driver

Start by:

• [ ] First running the NVIDIA sample binaries and being able to view the direct output of the GMSL camera feed
• [ ] Install and run ros_gmsl_driver, analyse and tweak parameters
• [ ] If it is still laggy, look into writing our own ros-gmsl driver interface (separate project)

[ZileiChen]

19/09/23 Goal: Mess around with ros and gmsl to hopefully find somewhere to start

Tried to ssh into Twizy using instructions from this page:

[SSH connection to the PX2](https://www.notion.so/SSH-connection-to-the-PX2-10f4de03594043eba232c09950e4817e?pvs=21)

Needed to configure ethernet connection, editing ipv4:

Ip Address: 192.168.1.3

Subnet Mask: 255.255.255.0

Default Gateway: left blank

Tried to reinstall ros gmsl driver using this command:

sudo apt install ros-kinetic-camera-info-manager

Gave some x11-common error

Solution:

Reinstall x11-common using sudo apt-get install —reinstall x11-common

Reconfigure it using sudo dpkg —configure -a

Next in the instructions was to run this command:

cd ~catkin_ws/src/

Gave file or directory not found error

Solution: create the required directory using these commands:

cd ~/ mkdir --parents catkin_ws/src cd catkin_ws catkin init

Gave catkin command not found error

Solution: install catkin in root using: sudo apt-get install ros-kinetic-catkin python-catkin-tools

Cloned ros_gmsl_driver git into catkin_ws/src directory using git clone https://github.com/BeckerFelix/ros_gmsl_driver

Installed ros kinetic to root, following the instructions from this link:

http://wiki.ros.org/kinetic/Installation/Ubuntu

• When neatening this log file, I forgot why we sidetracked to installing ros kinetic instead of continuing with the installation commands

Final line in the instructions to install ros gmsl driver is to running catkin_make in catkin_ws folder

This error is given

There is no /root/apt-repos/binary-DWx folder

Tried to install it, following instructions on:

https://docs.nvidia.com/drive/driveworks-3.5/dwx_samples_building_V5L.html

Cannot run sudo apt-get install cuda-cross-aarch64-10-2

Got tired and went home. Will try again later.

[dylan-gonzalez]

26/09/23 Driveworks include was located under /usr/local/driveworks/include/dw/Driveworks.h

changed the FindDriveworks.cmake file to have the actual location of the installed driveworks

• Initial code

  # Include dir
  find_path(Driveworks_INCLUDE_DIR
    NAMES dw/core/Version.h
    HINTS ${DRIVEWORKS_DIR}/targets/${CMAKE_SYSTEM_PROCESSOR}-linux/include
          ${Driveworks_PKGCONF_INCLUDE_DIRS}/../targets/${CMAKE_SYSTEM_PROCESSOR}-linux/include
    PATHS ${CMAKE_CURRENT_SOURCE_DIR}/../targets/${CMAKE_SYSTEM_PROCESSOR}-linux/include
          ${CMAKE_CURRENT_SOURCE_DIR}/../include
          ${DRIVEWORKS_DIR}/include
  )
  
  # Finally the library itself
  find_library(Driveworks_LIBRARY
    NAMES driveworks
    HINTS ${DRIVEWORKS_DIR}/targets/${CMAKE_SYSTEM_PROCESSOR}-linux/lib
          ${Driveworks_PKGCONF_LIBRARY_DIRS}/../targets/${CMAKE_SYSTEM_PROCESSOR}-linux/lib
    PATHS ${CMAKE_CURRENT_SOURCE_DIR}/../targets/${CMAKE_SYSTEM_PROCESSOR}-linux/lib
          ${CMAKE_CURRENT_SOURCE_DIR}/../lib
          ${DRIVEWORKS_DIR}/lib
  )

• After code

  # Include dir
  find_path(Driveworks_INCLUDE_DIR
    NAMES dw/core/Version.h
    HINTS /usr/local/driveworks/include
  )
  
  # Finally the library itself
  find_library(Driveworks_LIBRARY
    NAMES driveworks
    HINTS /usr/local
  )

The compilation got further but another error was given

• Error log

  root@tegra-ubuntu:~/catkin_ws# catkin_make
  Base path: /root/catkin_ws
  Source space: /root/catkin_ws/src
  Build space: /root/catkin_ws/build
  Devel space: /root/catkin_ws/devel
  Install space: /root/catkin_ws/install
  ####
  #### Running command: "cmake /root/catkin_ws/src -DCATKIN_DEVEL_PREFIX=/root/catkin_ws/devel -DCMAKE_INSTALL_PREFIX=/root/catkin_ws/install -G Unix Makefiles" in "/root/catkin_ws/build"
  ####
  -- Using CATKIN_DEVEL_PREFIX: /root/catkin_ws/devel
  -- Using CMAKE_PREFIX_PATH: /opt/ros/kinetic
  -- This workspace overlays: /opt/ros/kinetic
  -- Found PythonInterp: /usr/bin/python2 (found suitable version "2.7.12", minimum required is "2")
  -- Using PYTHON_EXECUTABLE: /usr/bin/python2
  -- Using Debian Python package layout
  -- Using empy: /usr/bin/empy
  -- Using CATKIN_ENABLE_TESTING: ON
  -- Call enable_testing()
  -- Using CATKIN_TEST_RESULTS_DIR: /root/catkin_ws/build/test_results
  -- Found gtest sources under '/usr/src/gmock': gtests will be built
  -- Found gmock sources under '/usr/src/gmock': gmock will be built
  -- Found PythonInterp: /usr/bin/python2 (found version "2.7.12")
  -- Using Python nosetests: /usr/bin/nosetests-2.7
  -- catkin 0.7.29
  -- BUILD_SHARED_LIBS is on
  -- BUILD_SHARED_LIBS is on
  -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  -- ~~  traversing 1 packages in topological order:
  -- ~~  - gmsl_n_cameras
  -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  -- +++ processing catkin package: 'gmsl_n_cameras'
  -- ==> add_subdirectory(ros_gmsl_driver)
  -- Cross Compiling for Vibrante
  -- Found Driveworks
  -- Found OpenCV: /opt/ros/kinetic (found version "3.3.1")
  Driveworks include directory /usr/local/driveworks/include/
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /root/catkin_ws/build
  ####
  #### Running command: "make -j6 -l6" in "/root/catkin_ws/build"
  ####
  Scanning dependencies of target gmsl_n_cameras_node
  [ 25%] Building CXX object ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/src/main.cpp.o
  [ 25%] Building CXX object ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/src/ProgramArguments.cpp.o
  [ 37%] Building CXX object ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/src/WindowEGL.cpp.o
  [ 50%] Building CXX object ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/src/WindowGLFW.cpp.o
  [ 75%] Building CXX object ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/src/SampleFramework.cpp.o
  [ 75%] Building CXX object ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/src/cv_connection.cpp.o
  [ 87%] Building CXX object ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/src/Log.cpp.o
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:119:5: error: ‘dwImageFormatConverterHandle_t’ does not name a type
       dwImageFormatConverterHandle_t yuv2rgba;
       ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:148:24: error: ‘dwImageFormatConverterHandle_t’ has not been declared
                          dwImageFormatConverterHandle_t yuv2rgba, NvMediaIJPE * );
                          ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp: In function ‘void threadCameraPipeline(Camera*, uint32_t, dwContextHandle_t, WindowBase*)’:
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:174:67: error: ‘DW_CAMERA_PROCESSED_IMAGE’ was not declared in this scope
           dwSensorCamera_getImageProperties(&cameraImageProperties, DW_CAMERA_PROCESSED_IMAGE,
                                                                     ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:177:32: error: ‘dwImageProperties {aka struct dwImageProperties}’ has no member named ‘pxlFormat’
           displayImageProperties.pxlFormat         = DW_IMAGE_RGBA;
                                  ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:177:52: error: ‘DW_IMAGE_RGBA’ was not declared in this scope
           displayImageProperties.pxlFormat         = DW_IMAGE_RGBA;
                                                      ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:178:32: error: ‘dwImageProperties {aka struct dwImageProperties}’ has no member named ‘planeCount’
           displayImageProperties.planeCount        = 1;
                                  ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:181:67: error: ‘struct Camera’ has no member named ‘yuv2rgba’
           result = dwImageFormatConverter_initialize(&cameraSensor->yuv2rgba, DW_IMAGE_NVMEDIA  , sdk);
                                                                     ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:181:100: error: ‘dwImageFormatConverter_initialize’ was not declared in this scope
           result = dwImageFormatConverter_initialize(&cameraSensor->yuv2rgba, DW_IMAGE_NVMEDIA  , sdk);
                                                                                                      ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:192:83: error: ‘dwImageNvMedia_create’ was not declared in this scope
                   result = dwImageNvMedia_create(&rgba, &displayImageProperties, sdk);
                                                                                     ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:259:51: error: ‘struct Camera’ has no member named ‘yuv2rgba’
                                       cameraSensor->yuv2rgba, cameraSensor->JPEGEncoders[cameraIdx]);
                                                     ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:295:55: error: ‘struct Camera’ has no member named ‘yuv2rgba’
           dwImageFormatConverter_release(&cameraSensor->yuv2rgba);
                                                         ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:295:63: error: ‘dwImageFormatConverter_release’ was not declared in this scope
           dwImageFormatConverter_release(&cameraSensor->yuv2rgba);
                                                                 ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:299:56: error: ‘dwImageNvMedia_destroy’ was not declared in this scope
           dwStatus result = dwImageNvMedia_destroy(&frame);
                                                          ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp: In function ‘void initSdk(dwContextObject**, WindowBase*)’:
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:581:27: error: ‘DW_VERSION’ was not declared in this scope
       dwInitialize(context, DW_VERSION, &sdkParams);
                             ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp: In function ‘void initSensors(std::vector<Camera>*, uint32_t*, dwSALHandle_t, ProgramArguments&)’:
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:669:51: error: ‘DW_CAMERA_PROCESSED_IMAGE’ was not declared in this scope
                                                     DW_CAMERA_PROCESSED_IMAGE,
                                                     ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp: At global scope:
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:703:24: error: ‘dwImageFormatConverterHandle_t’ has not been declared
                          dwImageFormatConverterHandle_t yuv2rgba,NvMediaIJPE *jpegEncoder)
                          ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp: In function ‘dwStatus captureCamera(dwImageNvMedia*, uint8_t*, dwSensorHandle_t, uint32_t, uint32_t, int, NvMediaIJPE*)’:
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:715:59: error: ‘DW_CAMERA_PROCESSED_IMAGE’ was not declared in this scope
       result = dwSensorCamera_getImageNvMedia(&frameNVMyuv, DW_CAMERA_PROCESSED_IMAGE, frameHandle);
                                                             ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:720:91: error: invalid conversion from ‘int’ to ‘dwContextHandle_t {aka dwContextObject*}’ [-fpermissive]
       result = dwImageFormatConverter_copyConvertNvMedia(frameNVMrgba, frameNVMyuv, yuv2rgba);
                                                                                             ^
  In file included from /usr/local/driveworks/include/dw/image/FormatConverter.h:94:0,
                   from /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:58:
  /usr/local/driveworks/include/dw/image/FormatConverter_vibrante.h:79:10: note:   initializing argument 3 of ‘dwStatus dwImageFormatConverter_copyConvertNvMedia(dwImageNvMedia*, const dwImageNvMedia*, dwContextHandle_t)’
   dwStatus dwImageFormatConverter_copyConvertNvMedia(dwImageNvMedia* output,
            ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp: In function ‘void renderFrame(dwImageStreamerHandle_t, dwRendererHandle_t)’:
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:754:60: error: ‘dwImageStreamer_receiveGL’ was not declared in this scope
       if (dwImageStreamer_receiveGL(&frameGL, 60000, streamer) != DW_SUCCESS) {
                                                              ^
  /root/catkin_ws/src/ros_gmsl_driver/src/main.cpp:761:59: error: ‘dwImageStreamer_returnReceivedGL’ was not declared in this scope
           dwImageStreamer_returnReceivedGL(frameGL, streamer);
                                                             ^
  ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/build.make:62: recipe for target 'ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/src/main.cpp.o' failed
  make[2]: *** [ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/src/main.cpp.o] Error 1
  make[2]: *** Waiting for unfinished jobs....
  CMakeFiles/Makefile2:469: recipe for target 'ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/all' failed
  make[1]: *** [ros_gmsl_driver/CMakeFiles/gmsl_n_cameras_node.dir/all] Error 2
  Makefile:138: recipe for target 'all' failed
  make: *** [all] Error 2
  Invoking "make -j6 -l6" failed

replaced this code dwImageFormatConverterHandle_t yuv2rgba; with dwImageHandle_t yuv2rgba;

using :%s/dwImageFormatConverterHandle_t/dwImageHandle_t/g in main.cpp

[dylan-gonzalez]

****Goal:**** Install Driveworks v5 onto px2

Installed sdkmanager_0.9.12-4180_amd64.deb onto local machine and then transferred it to px2, /home/nvidia

Ran sudo apt install ./sdkmanager_0.9.12-4180_amd64.deb but gave an error

• Error

  Reading package lists... Done
  Building dependency tree
  Reading state information... Done
  Note, selecting 'sdkmanager:amd64' instead of './sdkmanager_1.6.1-8175_amd64.deb'
  Some packages could not be installed. This may mean that you have
  requested an impossible situation or if you are using the unstable
  distribution that some required packages have not yet been created
  or been moved out of Incoming.
  The following information may help to resolve the situation:
  The following packages have unmet dependencies:
  sdkmanager:amd64 : Depends: libgconf-2-4:amd64 but it is not installable
  Depends: libcanberra-gtk-module:amd64 but it is not installable
  E: Unable to correct problems, you have held broken packages.

Well apparently we do this on our own machines then transfer over. Which worked a lot better now.

Had trouble using sdkmanager.

On one laptop, could not login on sdkmanager, developer login

Tried to reinstall, change browser. Did not work

Other laptop could login but the sdkmanager could not detect the drive platform. The product category was set to “Jetson” and could not be changed.

The laptop was connected to the px2 via usb. It was detected to actually be connected.

Back to the sdkmanager, it could not detect a target hardware hardware.

No idea where to go from here.

Potential reasons:

• Laptop was using ubuntu 22, not ubuntu 18 or lower
• Account reason but seems less likely