Intermittent errors while using `wait_for_frame()`

[nikhilnagaraj]

Describe the bug I am using the O3R225 - OVP800 combination to capture RGB and Pointcloud data. Ideally, I would like to capture images at random points in time (based on external inputs). However, the wait_for_frame function seems to be highly unreliable. Some of the errors that I have seen while invoking this function are:

• importlib._bootstrap.Error: No such file or directory
• importlib._bootstrap.Error: Invalid argument
• importlib._bootstrap.Error: Connection reset by peer

These errors occur more often while I try to capture data from the 3D port (they occur less frequently on the 2D port).

The VPU is not directly connected to the system where the scripts are executed but they are on the same network and I am consistently able to ping the device (thus likely eliminating network issues?).

This problem seems to vanish when I stream frames from the device using the on_next_frame(...) function.

Expected behavior wait_for_frame() should reliably return a frame.

Output/Screenshots If applicable, add console output and/or screenshots to help explain your problem.

To Reproduce

import asyncio
from random import random
import time
from ifm3dpy import O3R, FrameGrabber, buffer_id

o3r = O3R()
rgb_fg = FrameGrabber(o3r, 50011)
dist_fg = FrameGrabber(o3r, 50012)
rgb_fg.start([buffer_id.JPEG_IMAGE])
dist_fg.start([buffer_id.NORM_AMPLITUDE_IMAGE, buffer_id.RADIAL_DISTANCE_IMAGE, buffer_id.XYZ])

def reset_ports_config():
    config = o3r.get()
    config["ports"]["port1"]["state"] = "CONF"
    config["ports"]["port2"]["state"] = "CONF"
    o3r.set(config)
    config["ports"]["port1"]["state"] = "RUN"
    config["ports"]["port2"]["state"] = "RUN"
    o3r.set(config)

async def main():
    """Capture data from the sensor
    """
    reset_ports_config()
    while True:
        _ = await rgb_fg.wait_for_frame()
        _ = await dist_fg.wait_for_frame()
        print("Received frame...")
        # Sleep for a random interval from 0 - 5 seconds
        time.sleep(random() * 5)

if __name__ == "__main__":
    asyncio.run(main())

Configuration/Environment (please complete the following information):

• OS: WSL 2, Ubuntu 20.04.4 LTS
• Python Version: 3.10.4
• IFM3DPY Package Version: 1.0.0

Camera Configuration

{
  "device": {
    "clock": {
      "currentTime": 1664377930959807450
    },
    "diagnostic": {
      "temperatures": [
        {
          "entity": "PLL-therm",
          "value": 35.0
        },
        {
          "entity": "MCPU-therm",
          "value": 35.0
        },
        {
          "entity": "PMIC-Die",
          "value": 100.0
        },
        {
          "entity": "Tboard_tegra",
          "value": 31.0
        },
        {
          "entity": "GPU-therm",
          "value": 32.0
        },
        {
          "entity": "BCPU-therm",
          "value": 35.0
        },
        {
          "entity": "thermal-fan-est",
          "value": 33.79999923706055
        },
        {
          "entity": "Tdiode_tegra",
          "value": 30.5
        },
        {
          "entity": "port2",
          "value": 21
        }
      ],
      "upTime": 718513000000000
    },
    "info": {
      "description": "",
      "device": "0302",
      "deviceTreeBinaryBlob": "tegra186-quill-p3310-1000-c03-00-base.dtb",
      "guiSettings": "",
      "name": "",
      "partNumber": "OVP800",
      "productionState": "AA",
      "serialNumber": "000018384491",
      "vendor": "0001"
    },
    "log": {
      "level": "warn",
      "storage": "volatile"
    },
    "network": {
      "authorized_keys": "...",
      "interfaces": {
        "eth0": {
          "ipAddressConfig": 0,
          "ipv4": {
            "address": "...",
            "dns": "...",
            "gateway": "...",
            "mask": 22
          },
          "mac": "48:B0:2D:54:F6:DA",
          "networkSpeed": 10,
          "useDHCP": false
        },
        "eth1": {
          "ipAddressConfig": 1,
          "mac": "00:02:01:43:26:9F",
          "networkSpeed": 1000,
          "useDHCP": true
        }
      }
    },
    "state": {
      "errorMessage": "",
      "errorNumber": ""
    },
    "swVersion": {
      "euphrates": "1.4.3",
      "firmware": "0.14.23-493",
      "kernel": "4.9.140-l4t-r32.4+g1582a8a5405d",
      "l4t": "r32.4.3",
      "schema": "v0.2.4",
      "tcu": "1.0.2"
    }
  },
  "ports": {
    "port1": {
      "acquisition": {
        "framerate": 10.0,
        "version": {
          "major": 0,
          "minor": 0,
          "patch": 0
        }
      },
      "data": {
        "algoDebugConfig": {
          "channels": [
            {
              "id": "ov9782/ifcustom",
              "type": "normal"
            },
            {
              "id": "ov9782/profile",
              "type": "normal"
            },
            {
              "id": "ov9782/dbg",
              "type": "normal"
            },
            {
              "id": "ov9782/calib",
              "type": "once"
            },
            {
              "id": "ov9782/ifout",
              "type": "normal"
            },
            {
              "id": "ov9782/frameInfo",
              "type": "normal"
            }
          ],
          "version": {
            "major": 0,
            "minor": 11,
            "patch": 12
          }
        },
        "availablePCICOutput": [
          "RGB_INFO"
        ],
        "pcicTCPPort": 50011
      },
      "info": {
        "calibration": {
          "version": {
            "major": 0,
            "minor": 1,
            "patch": 1
          }
        },
        "device": "2301",
        "deviceTreeBinaryBlobOverlay": "001-ov9782.dtbo",
        "features": {
          "fov": {
            "horizontal": 127,
            "vertical": 80
          },
          "resolution": {
            "height": 800,
            "width": 1280
          },
          "type": "2D"
        },
        "icc": {
          "version": {
            "major": 0,
            "minor": 0,
            "patch": 0
          }
        },
        "name": "",
        "partNumber": "O3R225",
        "productionState": "AA",
        "sensor": "OV9782",
        "sensorID": "OV9782_127x80_noIllu_Csample",
        "serialNumber": "000018803591",
        "vendor": "0001",
        "version": {
          "major": 11,
          "minor": 1,
          "patch": 0
        }
      },
      "mode": "experimental_autoexposure2D",
      "processing": {
        "extrinsicHeadToUser": {
          "rotX": 0.0,
          "rotY": 0.0,
          "rotZ": 0.0,
          "transX": 0.0,
          "transY": 0.0,
          "transZ": 0.0
        },
        "version": {
          "major": 0,
          "minor": 11,
          "patch": 12
        }
      },
      "state": "RUN"
    },
    "port2": {
      "acquisition": {
        "channelSelection": "manual",
        "channelValue": 0,
        "exposureLong": 5000,
        "exposureShort": 400,
        "framerate": 10.0,
        "offset": 0.0,
        "version": {
          "major": 0,
          "minor": 0,
          "patch": 0
        }
      },
      "data": {
        "algoDebugConfig": {
          "channels": [
            {
              "id": "irs2381/ifcustom",
              "type": "normal"
            },
            {
              "id": "irs2381/profile",
              "type": "normal"
            },
            {
              "id": "irs2381/dbg",
              "type": "normal"
            },
            {
              "id": "irs2381/calib",
              "type": "once"
            },
            {
              "id": "irs2381/ifout_compr",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/mode",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/0",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/1",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/2",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/3",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/4",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/5",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/6",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/7",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/8",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/9",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/10",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/11",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/12",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/13",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/14",
              "type": "normal"
            },
            {
              "id": "irs2381/raw/15",
              "type": "normal"
            }
          ],
          "version": {
            "major": 0,
            "minor": 11,
            "patch": 12
          }
        },
        "availablePCICOutput": [
          "TOF_INFO",
          "RADIAL_DISTANCE_NOISE",
          "RADIAL_DISTANCE_COMPRESSED",
          "REFLECTIVITY",
          "AMPLITUDE_COMPRESSED",
          "CONFIDENCE"
        ],
        "pcicTCPPort": 50012
      },
      "info": {
        "calibration": {
          "version": {
            "major": 0,
            "minor": 5,
            "patch": 1
          }
        },
        "device": "3101",
        "deviceTreeBinaryBlobOverlay": "001-irs2381c.dtbo",
        "features": {
          "fov": {
            "horizontal": 105,
            "vertical": 78
          },
          "resolution": {
            "height": 172,
            "width": 224
          },
          "type": "3D"
        },
        "icc": {
          "version": {
            "major": 1,
            "minor": 0,
            "patch": 4
          }
        },
        "name": "",
        "partNumber": "O3R225",
        "productionState": "AA",
        "sensor": "IRS2381C",
        "sensorID": "IRS2381C_105x78_4x2W_110x90_C7",
        "serialNumber": "000018803591",
        "vendor": "0001",
        "version": {
          "major": 11,
          "minor": 1,
          "patch": 0
        }
      },
      "mode": "standard_range4m",
      "processing": {
        "diParam": {
          "anfFilterSizeDiv2": 2,
          "dynamicSymmetryThreshold": 20.0,
          "enableTemporalFilter": true,
          "maxDistNoise": 0.05,
          "medianSizeDiv2": 0,
          "minAmplitude": 20.0,
          "minReflectivity": 2.0,
          "mixedPixelFilterMode": 1,
          "mixedPixelThresholdRad": 0.05
        },
        "extrinsicHeadToUser": {
          "rotX": 0.0,
          "rotY": 0.0,
          "rotZ": 0.0,
          "transX": 0.0,
          "transY": 0.0,
          "transZ": 0.0
        },
        "version": {
          "major": 0,
          "minor": 11,
          "patch": 12
        }
      },
      "state": "RUN"
    }
  }
}

Additional context Add any other context about the problem here.

[lola-masson]

Hi @nikhilnagaraj, I just tried your script and have not yet seen any of these errors you are listing. Do you have an idea of how long the script has to run before you see the issues?

The wait_for_frame() is not going to perform well in a while True loop but I would expect that with the sleep() you put in there it should be sufficient to not run into any delay.

Are you able to give us more details on your network connection? Which ethernet port are you using (it looks like eth0 from the json config but just want to confirm)? Are you connected through a router, wifi, etc?

[nikhilnagaraj]

Hi @lola-masson It usually occurs within the first 20-25 calls. I suspected that it was a network issue, but the thrown errors made me think otherwise.

Regarding the network connection, I am using eth0 and the VPU is on the same network as I am (not connected directly). I am able to ping the VPU and receive frames semi-regularly, so I am unsure if this is an issue with the network. However, in a few days, I will be testing this with a physical cable connecting the VPU to my device (or by pushing a docker image to the VPU) so I should have better feedback.

Might the issue of having to reset the RGB imagers to CONF and then RUN as listed here be related to this? I have definitely seen that I have to reset the RGB imagers to be able to receive any data if the delay between subsequent frame requests is long enough. (Similar issue with the distance imagers, where changing the mode to standard_range2m and then back to standard_range4m seems to help).

[lola-masson]

It seems like this might be a bug on our end. The fact that you have to "restart" each imager by switching is CONF->RUN or standard_range2m->standard_range4m is strange, this should not happen and we are investigating. We will let you know as soon as we find the issue and provide a solution.

[nikhilnagaraj]

@lola-masson Any updates on this?

[desengph]

@nikhilnagaraj can you please contact us via email at: support.robotics@ifm.com

We tried to reach you via your linked email address on your profile on GitHub, but the emails can't be delivered.

[nikhilnagaraj]

@nikhilnagaraj can you please contact us via email at: support.robotics@ifm.com

We tried to reach you via your linked email address on your profile on GitHub, but the emails can't be delivered.

Done! :)

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.