Empty point cloud received from O3D303

[robinbansal08]

I am trying to get point cloud data from O3D303 sensor from company's system (already disabled the proxy for the ip address of the O3D303 sensor). I have used a simple sample code to get captured frames from the O3D303 sensor, but the point cloud I receive is empty. I tried with amplitude and depth image, amplitude image is also empty but distance image has depth data. This issue is particular to the system provided by my company installed with Ubuntu 20.04. In my private system (Ubuntu 22.04) everything is working fine. The code I am using is:

    ifm3d::Camera::Ptr cam = std::make_shared<ifm3d::Camera>(sensor_ip);
    ifm3d::ImageBuffer::Ptr img = std::make_shared<ifm3d::ImageBuffer>();
    ifm3d::FrameGrabber::Ptr fg = std::make_shared<ifm3d::FrameGrabber>(cam, ifm3d::IMG_AMP|ifm3d::IMG_RDIS|ifm3d::IMG_CART);
    pcl::PointCloud<pcl::PointXYZI>::Ptr cloud (new pcl::PointCloud<pcl::PointXYZI>);
    std::cout<<"Sending software trigger to the camera\n";
    fg->SWTrigger();  
    if (! fg->WaitForFrame(img.get(), 1000))
    {
      std::cerr << "Timeout waiting for camera !!" << std::endl;
      return -1;
    }
    pcl::io::savePCDFileASCII("point_cloud.pcd", *(img->Cloud()));
    imwrite("amplitude.png", img->AmplitudeImage());
    imwrite("radial_distance.png", img->DistanceImage());

The file using ifm3d dump:

{
  "ifm3d": {
    "Apps": [
      {
        "Description": "",
        "Id": "1701072507",
        "Imager": {
          "AutoExposureMaxExposureTime": "10000",
          "AutoExposureReferencePointX": "88",
          "AutoExposureReferencePointY": "66",
          "AutoExposureReferenceROI": "{\"ROIs\":[{\"id\":0,\"group\":0,\"type\":\"Rect\",\"width\":130,\"height\":100,\"angle\":0,\"center_x\":88,\"center_y\":66}]}",
          "AutoExposureReferenceType": "0",
          "Channel": "0",
          "ClippingBottom": "263",
          "ClippingCuboid": "{\"XMax\":3.4028229999999999e+38,\"XMin\":-3.4028229999999999e+38,\"YMax\":3.4028229999999999e+38,\"YMin\":-3.4028229999999999e+38,\"ZMax\":3.4028229999999999e+38,\"ZMin\":-3.4028229999999999e+38}",
          "ClippingLeft": "0",
          "ClippingRight": "351",
          "ClippingTop": "0",
          "ContinuousAutoExposure": "false",
          "ContinuousUserFrameCalibration": "false",
          "EnableAmplitudeCorrection": "true",
          "EnableFastFrequency": "false",
          "EnableFilterAmplitudeImage": "true",
          "EnableFilterDistanceImage": "true",
          "EnableRectificationAmplitudeImage": "false",
          "EnableRectificationDistanceImage": "false",
          "ExposureTime": "10000",
          "ExposureTimeList": "5000;10000",
          "ExposureTimeRatio": "2",
          "FlyingPixelEraser": "0",
          "FrameRate": "5",
          "MaxAllowedLEDFrameRate": "8.1",
          "MinimumAmplitude": "42",
          "Resolution": "1",
          "SpatialFilter": {},
          "SpatialFilterType": "0",
          "SymmetryThreshold": "0.4",
          "TemporalFilter": {},
          "TemporalFilterType": "0",
          "ThreeFreqMax2FLineDistPercentage": "80",
          "ThreeFreqMax3FLineDistPercentage": "80",
          "TwoFreqMaxLineDistPercentage": "80",
          "Type": "under5m_moderate",
          "UseSimpleBinning": "false"
        },
        "Index": "1",
        "LogicGraph": "{\"IOMap\": {\"OUT1\": \"RFT\",\"OUT2\": \"AQUFIN\"},\"blocks\": {\"B00001\": {\"pos\": {\"x\": 200,\"y\": 200},\"properties\": {},\"type\": \"PIN_EVENT_IMAGE_ACQUISITION_FINISHED\"},\"B00002\": {\"pos\": {\"x\": 200,\"y\": 75},\"properties\": {},\"type\": \"PIN_EVENT_READY_FOR_TRIGGER\"},\"B00003\": {\"pos\": {\"x\": 600,\"y\": 75},\"properties\": {\"pulse_duration\": 0},\"type\": \"DIGITAL_OUT1\"},\"B00005\": {\"pos\": {\"x\": 600,\"y\": 200},\"properties\": {\"pulse_duration\": 0},\"type\": \"DIGITAL_OUT2\"}},\"connectors\": {\"C00000\": {\"dst\": \"B00003\",\"dstEP\": 0,\"src\": \"B00002\",\"srcEP\": 0},\"C00001\": {\"dst\": \"B00005\",\"dstEP\": 0,\"src\": \"B00001\",\"srcEP\": 0}}}",
        "Name": "Sample Application",
        "PcicEipResultSchema": "{ \"layouter\": \"flexible\", \"format\": { \"dataencoding\": \"binary\", \"order\": \"big\" }, \"elements\" : [ { \"type\": \"string\", \"value\": \"star\", \"id\": \"start_string\" }, { \"type\": \"records\", \"id\": \"models\", \"elements\": [ { \"type\": \"int16\", \"id\": \"boxFound\" }, { \"type\": \"int16\", \"id\": \"width\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"height\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"length\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"xMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"yMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"zMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"yawAngle\" }, { \"type\": \"int16\", \"id\": \"qualityLength\" }, { \"type\": \"int16\", \"id\": \"qualityWidth\" }, { \"type\": \"int16\", \"id\": \"qualityHeight\" } ] }, { \"type\": \"string\", \"value\": \"stop\", \"id\": \"end_string\" } ] }",
        "PcicPnioResultSchema": "{\"layouter\" : \"flexible\", \"format\": { \"dataencoding\": \"binary\", \"order\": \"big\" }, \"elements\" : [ { \"type\": \"string\", \"value\": \"star\", \"id\": \"start_string\" }, { \"type\": \"records\", \"id\": \"models\", \"elements\": [ { \"type\": \"int16\", \"id\": \"boxFound\" }, { \"type\": \"int16\", \"id\": \"width\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"height\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"length\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"xMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"yMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"zMidTop\", \"format\": { \"scale\": 1000 } }, { \"type\": \"int16\", \"id\": \"yawAngle\" }, { \"type\": \"int16\", \"id\": \"qualityLength\" }, { \"type\": \"int16\", \"id\": \"qualityWidth\" }, { \"type\": \"int16\", \"id\": \"qualityHeight\" } ] }, { \"type\": \"string\", \"value\": \"stop\", \"id\": \"end_string\" } ] }",
        "PcicTcpResultSchema": "{ \"layouter\": \"flexible\", \"format\": { \"dataencoding\": \"ascii\" }, \"elements\": [ { \"type\": \"string\", \"value\": \"star\", \"id\": \"start_string\" }, { \"type\": \"blob\", \"id\": \"normalized_amplitude_image\" }, { \"type\": \"blob\", \"id\": \"distance_image\" }, { \"type\": \"blob\", \"id\": \"x_image\" }, { \"type\": \"blob\", \"id\": \"y_image\" }, { \"type\": \"blob\", \"id\": \"z_image\" }, { \"type\": \"blob\", \"id\": \"confidence_image\" }, { \"type\": \"blob\", \"id\": \"diagnostic_data\" }, { \"type\": \"string\", \"value\": \"stop\", \"id\": \"end_string\" } ] }",
        "RtspOverlayStyle": "{\"ROI\": {\"default\": {\"visible\": true, \"filled\": false, \"use_symbol\": false, \"label_alignment\": \"top\", \"label_content\": \"\", \"label_background\": \"black\", \"label_fontsize\": 8, \"label_failonly\": false}, \"model_defaults\": {}, \"specific\": {} } }",
        "TemplateInfo": "",
        "TriggerMode": "2",
        "Type": "Camera"
      }
    ],
    "Device": {
      "ActiveApplication": "1",
      "ArticleNumber": "O3D303",
      "ArticleStatus": "AD",
      "Description": "",
      "DeviceType": "1:2",
      "EIPConsumingSize": "8",
      "EIPProducingSize": "450",
      "EnableAcquisitionFinishedPCIC": "false",
      "EthernetFieldBus": "0",
      "EthernetFieldBusEndianness": "0",
      "EvaluationFinishedMinHoldTime": "10",
      "ExtrinsicCalibRotX": "0",
      "ExtrinsicCalibRotY": "0",
      "ExtrinsicCalibRotZ": "0",
      "ExtrinsicCalibTransX": "0",
      "ExtrinsicCalibTransY": "0",
      "ExtrinsicCalibTransZ": "0",
      "IODebouncing": "true",
      "IOExternApplicationSwitch": "0",
      "IOLogicType": "1",
      "IPAddressConfig": "0",
      "ImageTimestampReference": "946686038",
      "Name": "ifm3d unit test",
      "OperatingMode": "0",
      "PNIODeviceName": "",
      "PasswordActivated": "false",
      "PcicProtocolVersion": "3",
      "PcicTcpPort": "50010",
      "PcicTcpSchemaAutoUpdate": "false",
      "SaveRestoreStatsOnApplSwitch": "true",
      "ServiceReportFailedBuffer": "15",
      "ServiceReportPassedBuffer": "15",
      "SessionTimeout": "30",
      "TemperatureFront1": "3276.7",
      "TemperatureFront2": "3276.7",
      "TemperatureIMX6": "31.3840007781982",
      "TemperatureIllu": "36.7",
      "UpTime": "0.347222222222222"
    },
    "Net": {
      "MACAddress": "00:02:01:41:3B:B2",
      "NetworkSpeed": "0",
      "StaticIPv4Address": "192.168.0.69",
      "StaticIPv4Gateway": "192.168.0.201",
      "StaticIPv4SubNetMask": "255.255.255.0",
      "UseDHCP": "false"
    },
    "Time": {
      "CurrentTime": "946686036",
      "MaxNumberOfNTPServers": "5",
      "NTPServers": "",
      "StartingSynchronization": "false",
      "Stats": "",
      "SynchronizationActivated": "false",
      "Syncing": "false",
      "WaitSyncTries": "1"
    },
    "_": {
      "Date": "Thu Jun 29 11:11:00 2023",
      "HWInfo": {
        "Connector": "#!02_A300_C40_04246016_008023176",
        "Diagnose": "#!02_D322_C34_04250651_008026824",
        "Frontend": "#!02_F342_C34_19_00105_008023607",
        "Illumination": "#!02_I300_001_04201465_008001175",
        "MACAddress": "00:02:01:41:3B:B2",
        "Mainboard": "#!02_M381_C41_03901679_008023690",
        "MiraSerial": "0070-81eb-0e63-0688"
      },
      "SWVersion": {
        "Algorithm_Version": "3.0.5",
        "Calibration_Device": "00:02:01:41:3b:b2",
        "Calibration_Version": "1.0.2",
        "Diagnostic_Controller": "O2I_1.20.7428-0-g91c60a74be",
        "IFM_Recovery": "unversioned",
        "IFM_Software": "1.71.7444",
        "Linux": "Linux version 3.14.34-rt31-yocto-standard-00048-gc8112a91aeb2-d",
        "Main_Application": "O2I_1.20.7428-0-g91c60a74be"
      },
      "ifm3d_version": 2003
    }
  }
}

[graugans]

Can you provide more details what empty means. The file size is 0 bytes or the matrix is filled with 0 values

[robinbansal08]

I believe that file size is 0 bytes because during point cloud save method pcl::io::savePCDFileASCII("point_cloud.pcd", *(img->Cloud())) I am getting error terminate called after throwing an instance of 'pcl::IOException' what(): : [pcl::PCDWriter::writeASCII] Input point cloud has no data! Aborted

[robinbansal08]

Update: Amplitude image: It is non-empty. Visually it looked black but I just checked the values of each pixel, it is non-zero. Point Cloud: I checked again regarding size of point cloud received, it is 0, confirmed with size() method.

cout << "Size of point cloud: " << img->Cloud()->size() << endl;

[lola-masson]

Hi @robinbansal08,

Are you getting data if you call XYZImage() instead of Cloud()?

[robinbansal08]

Yes, I am getting data for XYZImage()

[graugans]

How did you install ifm3d on both of the systems? Are both versions compiled from source?

[robinbansal08]

In both of the systems I have installed ifm3d from source.

[robinbansal08]

I tried to search O3D303 camera in local network using https://github.com/ifm/python-ipv4-discovery/blob/master/discovery.py. It is failing with error "Could not find ipv4 discovery server timed out".

$ sudo tcpdump -nnvXSs 0  -i wlan0 udp port 3321
tcpdump: listening on wlan0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:22:15.163918 IP (tos 0x0, ttl 64, id 37682, offset 0, flags [DF], proto UDP (17), length 36)
    192.168.0.118.3321 > 255.255.255.255.3321: UDP, length 8
    0x0000:  4500 0024 9332 4000 4011 e678 c0a8 0076  E..$.2@.@..x...v
    0x0010:  ffff ffff 0cf9 0cf9 0010 c13f 1020 efcf  ...........?....
    0x0020:  0cf9 0000                                ....
12:22:15.248796 IP (tos 0x10, ttl 16, id 0, offset 0, flags [DF], proto UDP (17), length 388)
    0.0.0.0.3321 > 255.255.255.255.3321: UDP, length 360
    0x0000:  4510 0184 0000 4000 1011 695a 0000 0000  E.....@...iZ....
    0x0010:  ffff ffff 0cf9 0cf9 0170 0a95 1911 1981  .........p......
    0x0020:  c0a8 0045 c0a8 00c9 ffff ff00 1f90 0001  ...E............
    0x0030:  0002 0000 0000 0000 0000 0000 0002 0141  ...............A
    0x0040:  3bb2 0008 4f33 4433 3033 2d34 312d 3342  ;...O3D303-41-3B
    0x0050:  2d42 3200 0000 0000 0000 0000 0000 0000  -B2.............
    0x0060:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0070:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0080:  0000 0000 6966 6d33 6420 756e 6974 2074  ....ifm3d.unit.t
    0x0090:  6573 7400 0000 0000 0000 0000 0000 0000  est.............
    0x00a0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x00b0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x00c0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x00d0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x00e0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x00f0:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0100:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0110:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0120:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0130:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0140:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0150:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0160:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0170:  0000 0000 0000 0000 0000 0000 0000 0000  ................
    0x0180:  0000 0000                                ....

In tcpdump output, we can see the broadcast reply from camera but this is not received back by the discovery application. Do you think it might be some firewall issue (also the reason for receiving empty point cloud) as I am successfully able to discover the O3D303 camera using my private computer but in company's computer ?

[lola-masson]

In theory, the Cloud image is just the XYZImage repackaged into a PCL container, so if you are receiving XYZ I don't think there is a connection issue. The fact that the same code is working fine on a different machine makes me suspect an installation issue. Maybe a missing dependency to PCL? Did you install all the build dependencies as specified in https://github.com/ifm/ifm3d/blob/v0.20.3/doc/source_build.md#build-dependencies? I would expect that the build fails if you are missing dependencies, but it worth checking nonetheless.

[robinbansal08]

The issue is resolved with re-installation of ifm3d library. I am able to receive point cloud successfully. Thanks a lot for your support.