ApolloAuto / apollo

An open autonomous driving platform
Apache License 2.0
24.82k stars 9.65k forks source link

Inconsistency detected by ld.so #15142

Open IvDmNe opened 10 months ago

IvDmNe commented 10 months ago

On Apollo 8.0 I was experimenting on combining different components in one process to reduce message transport time. I found a bug: when I have running static_transform module running and try to run another dag file with some components, the latter one fails with the following message:

Inconsistency detected by ld.so: ../elf/dl-tls.c: 517: _dl_allocate_tls_init: Assertion `listp != NULL' failed!

Steps to reproduce:

  1. Start static transform: mainboard -d /apollo/modules/transform/dag/static_transform.dag;
  2. Start dag file with lidar fusion + compensator + lidar detection + tracking, like this one:
module_config {
    module_library : "/apollo/bazel-bin/modules/drivers/lidar/velodyne/fusion/libvelodyne_fusion_component.so"

    components {
      class_name : "PriSecFusionComponent"
      config {
        name : "velodyne_fusion"
        config_file_path : "/apollo/modules/drivers/lidar/velodyne/conf/velodyne_fusion_conf.pb.txt"
        readers {channel: "/apollo/sensor/lidar16/front/center/PointCloud2"}
      }
    }
}

module_config {
    module_library : "/apollo/bazel-bin/modules/drivers/lidar/velodyne/compensator/libvelodyne_compensator_component.so"

    components {
      class_name : "CompensatorComponent"
      config {
        name : "velodyne16_fusion_compensator"
        config_file_path : "/apollo/modules/drivers/lidar/velodyne/conf/velodyne16_fusion_compensator.pb.txt"
        readers {channel: "/apollo/sensor/lidar16/fusion/PointCloud2"}
      }
    }
}

module_config {
  module_library : "/apollo/bazel-bin/modules/perception/onboard/component/libperception_component_lidar.so"

  components {
    class_name : "LidarDetectionComponent"
    config {
      name: "Velodyne128Detection"
      config_file_path: "/apollo/modules/perception/production/conf/perception/lidar/velodyne128_detection_conf.pb.txt"
      flag_file_path: "/apollo/modules/perception/production/conf/perception/perception_common.flag"
      readers {
        channel: "/apollo/sensor/lidar128/compensator/PointCloud2"
      }
    }
  }

  components {
    class_name : "LidarTrackingComponent"
    config {
      name: "LidarTrackingComponent"
      config_file_path: "/apollo/modules/perception/production/conf/perception/lidar/recognition_conf.pb.txt"
      readers {
        channel: "/perception/inner/DetectionObjects"
      }
    }
  }

  components {
    class_name: "MultiSensorFusionComponent"
    config {
      name: "SensorFusion"
      config_file_path: "/apollo/modules/perception/production/conf/perception/fusion/fusion_component_conf.pb.txt"
      readers {
        channel: "/perception/inner/PrefusedObjects"
      }
    }
  }
}

module_config {
  module_library : "/apollo/bazel-bin/modules/v2x/fusion/apps/libv2x_fusion_component.so"

  components {
    class_name : "V2XFusionComponent"
    config {
      name : "v2x_fusion"
      flag_file_path : "/apollo/modules/v2x/conf/v2x_fusion_tracker.conf"
      readers: [
        {
          channel: "/perception/vehicle/obstacles"
        }
      ]
    }
  }
}

The error does not occur when we run these dag files in reverse order as well as if we merge contents of the files into one dag.

It seems that this is a bug in glibc. similar Issue.

Have anyone tried to deal with it?

daohu527 commented 9 months ago

yes, we already fixed it. We only have this problem on arm. The solution is to upgrade gcc or apply a patch.

IvDmNe commented 8 months ago

Can you please specify version of gcc? I tried 9.4 (installed with scripts/install_gcc.sh script) - still facing this error. With gcc 10.3 GPU modules do not compile (seems that there is a bug in this version with CUDA).

sinopec commented 3 weeks ago

yes, we already fixed it. We only have this problem on arm. The solution is to upgrade gcc or apply a patch.

I Got the sasme probem in apollo 9.0 pkg mode. Seems not solved yet. Can you give a possible solution. CPU: 11th Gen Intel(R) Core(TM) i7-1185GRE @ 2.80GHz host os : ubuntu 22.04 docker image : registry.baidubce.com/apollo/apollo-env-cpu:9.0-latest