Got [Script(6)] [critical] KeyError: ('frame',) when running gen2-face-recognition

[FrancisTse8]

I got the following error when running main.py in gen2-face-recognition. I copied all the files from the folder and checked that I met all the requirements. I am running depthai 2.19.1.0, blobconverter 1.3.0, nmpy 1.22.2 and opncv-python 4.5.5.62. Did I do something wrong? Thanks for any pointers to resolve the error.

francis@raspberrypi:~/Desktop/learningOAK-D-Lite $ /bin/python /home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py
Creating pipeline...
Creating Color Camera...
Creating Face Detection Neural Network...
Creating Head pose estimation NN
/home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py:164: DeprecationWarning: Use 'inputConfig.setWaitForMessage()' instead
  headpose_manip.setWaitForConfigInput(True)
Downloading /home/francis/.cache/blobconverter/head-pose-estimation-adas-0001_openvino_2021.4_6shave.blob...
[==================================================]
Done
Creating face recognition ImageManip/NN
/home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py:179: DeprecationWarning: Use 'inputConfig.setWaitForMessage()' instead
  face_rec_manip.setWaitForConfigInput(True)
Downloading /home/francis/.cache/blobconverter/face-recognition-arcface-112x112_openvino_2021.4_6shave.blob...
[==================================================]
Done
[18443010F1AECE1200] [1.1.1] [2.090] [Script(6)] [critical] KeyError: ('frame',)

At:
  <script>(18): <module>

Traceback (most recent call last):
  File "/home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py", line 206, in <module>
    if q.has():
RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'color' (X_LINK_ERROR)'

[Erol444]

Hi @FrancisTse8 , I just tried it out with latest depthai (2.20.1) and latest depthai-experiments repo commit (4dd9dd7c354921eb7998f50f35ff1da76ca603dc) and it works as expected. Could you check versions/commit you are using? Thanks, Erik

[FrancisTse8]

Yes, I am running the code from the latest code (4dd9dd7c354921eb7998f50f35ff1da76ca603dc) from the depthai-experiments. Since you used a later depthai, I upgraded my depthai to the latest which turned out to be 2.20.2. However, when I ran the main.py code, I still got the error:

francis@raspberrypi:~/Desktop/learningOAK-D-Lite $ /bin/python /home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py
Creating pipeline...
Creating Color Camera...
Creating Face Detection Neural Network...
Creating Head pose estimation NN
/home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py:164: DeprecationWarning: Use 'inputConfig.setWaitForMessage()' instead
  headpose_manip.setWaitForConfigInput(True)
Creating face recognition ImageManip/NN
/home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py:179: DeprecationWarning: Use 'inputConfig.setWaitForMessage()' instead
  face_rec_manip.setWaitForConfigInput(True)
[18443010F1AECE1200] [1.1.1] [2.137] [Script(6)] [critical] KeyError: ('frame',)

At:
  <script>(18): <module>

Traceback (most recent call last):
  File "/home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py", line 206, in <module>
    if q.has():
RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'color' (X_LINK_ERROR)'

BTW, I am running on a Raspberry Pi 4 with Bullseye if that makes a difference:

francis@raspberrypi:~/Desktop/learningOAK-D-Lite $ cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/

[Erol444]

Hi @FrancisTse8 , Perhaps it's because of the lower bandwidth (maybe usb2?) and frame doesn't arrive to the host (yet). I couldn't reproduce locally, but I did create a check for that (commit here). Could you try updating the depthai-experiments? As I couldn't test it might not fix the issue, it's just my best guess. Thanks, Erik

[FrancisTse8]

I changed line 18 on scripts.py as: if str(target_seq) in sync and "frame" in sync[str(target_seq)]: but still have same problem.

BTW, the OAK-D-Lite has been plugged in to the USB 3 port on the Raspberry Pi 4.

[Erol444]

Did you perhaps change the code? This error msg doesn't look right...

[FrancisTse8]

I did change the code on scripts.py. On line 18 on scripts.py, it was if str(target_seq) in sync:. I changed it to if str(target_seq) in sync and "frame" in sync[str(target_seq)]:. Please confirm that this is that update that you wanted me to try.

After that, I still got the error

francis@raspberrypi:~/Desktop/learningOAK-D-Lite $ /bin/python /home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py
Creating pipeline...
Creating Color Camera...
Creating Face Detection Neural Network...
Creating Head pose estimation NN
/home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py:164: DeprecationWarning: Use 'inputConfig.setWaitForMessage()' instead
  headpose_manip.setWaitForConfigInput(True)
Creating face recognition ImageManip/NN
/home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py:179: DeprecationWarning: Use 'inputConfig.setWaitForMessage()' instead
  face_rec_manip.setWaitForConfigInput(True)
[2023-02-01 16:52:22.568] [warning] Skipping X_LINK_UNBOOTED device with name "1.1.1" ()
[18443010F1AECE1200] [1.1.1] [2.158] [Script(6)] [critical] KeyError: ('frame',)

At:
  <script>(18): <module>

Traceback (most recent call last):
  File "/home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py", line 206, in <module>
    if q.has():
RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'color' (X_LINK_ERROR)'

[FrancisTse8]

Hello @Erol444,

I saw the changes you made on commit 8990bfc4640835d78e4c7d89aa8e1cd8247a3bf1 today. So, I copied the changed main.py and script.py files to my local directory and ran them. These time I did not errors anymore. The image window came up and it showed two faces from a picture I put in front of the OAK-D-Lite.

It turned out that the new script.py from today's release are the same as the changes I made on the scipt.py file yesterday. Out of curiosity, I brought back the script.py file I used yesterday and ran it again and it did not produce error. As a matter of fact, I went back to the original commit (4dd9dd7c354921eb7998f50f35ff1da76ca603dc), and it ran without error also.

I did not change anything on the hardware side. Did not plug and unplug anything except for powering off the Pi 4 overnight. Kind of strange but I am glad it is now working so I can continue my learning and testing.

Thanks for all your help.

[FrancisTse8]

Hello @Erol444,

Sorry to have to reopen the issue. I stopped working with gen2-face-recognition for a few days to work on some other pipelines. Today, when I ran the program again, I got the same error as before. I tried doing a complete system power down and power up a few times but still got the same error. Looking at the script.py code, there are a few node.warn() statements that were commented out. So, I uncommented them. I added one for the situation if preview is not None:. The following is the complete script.py code that I am using. There should not be any changes except for the node.warn() statements:

import time
sync = {} # Dict of messages

# So the correct frame will be the first in the list
# For this experiment this function is redundant, since everything
# runs in blocking mode, so no frames will get lost
def get_sync(target_seq):
    seq_remove = [] # Arr of sequence numbers to get deleted
    for seq, msgs in sync.items():
        if seq == str(target_seq):
            # We have synced msgs, remove previous msgs (memory cleaning)
            for rm in seq_remove:
                del sync[rm]
            return msgs
        seq_remove.append(seq) # Will get removed from dict if we find synced sync pair
    return None
def find_frame(target_seq):
    #node.warn(f"find_frame start")
    if str(target_seq) in sync and "frame" in sync[str(target_seq)]:
        return sync[str(target_seq)]["frame"]
def add_detections(det, seq):
    # No detections, we can remove saved frame
    if len(det) == 0:
        del sync[str(seq)]
    else:
        # Save detections, as we will need them for face recognition model
        sync[str(seq)]["detections"] = det

def correct_bb(bb):
    if bb.xmin < 0: bb.xmin = 0.001
    if bb.ymin < 0: bb.ymin = 0.001
    if bb.xmax > 1: bb.xmax = 0.999
    if bb.ymax > 1: bb.ymax = 0.999

while True:
    time.sleep(0.001)
    preview = node.io['preview'].tryGet()
    if preview is not None:
        node.warn(f"New frame start")
        sync[str(preview.getSequenceNum())] = {}
        sync[str(preview.getSequenceNum())]["frame"] = preview

    face_dets = node.io['face_det_in'].tryGet()
    if face_dets is not None:
        node.warn(f"New detection start")
        passthrough = node.io['face_pass'].get()
        seq = passthrough.getSequenceNum()
        node.warn(f"New detection {seq}")
        if len(sync) == 0: continue
        img = find_frame(seq) # Matching frame is the first in the list
        if img is None: continue

        add_detections(face_dets.detections, seq)

        for det in face_dets.detections:
            cfg = ImageManipConfig()
            correct_bb(det)                                                # clip det bb values to within 0.001 and 0.999
            cfg.setCropRect(det.xmin, det.ymin, det.xmax, det.ymax)
            cfg.setResize(60, 60)
            cfg.setKeepAspectRatio(False)
            node.io['manip_cfg'].send(cfg)                                  # send detected faces bb to crop to and resize to 60x60
            node.io['manip_img'].send(img)                                  # send out preview frame corresponding to the detection seq number

    headpose = node.io['headpose_in'].tryGet()
    if headpose is not None:
        node.warn(f"New headpose")
        passthrough = node.io['headpose_pass'].get()
        seq = passthrough.getSequenceNum()
        node.warn(f"New headpose seq {seq}")
        # Face rotation in degrees
        r = headpose.getLayerFp16('angle_r_fc')[0] # Only 1 float in there

        msgs = get_sync(seq)                                                # get info for this headpose detection and remove all previous items on sync
        bb = msgs["detections"].pop(0)
        correct_bb(bb)

        # remove_prev_frame(seq)
        img = msgs["frame"]
        # node.warn('HP' + str(img))
        # node.warn('bb' + str(bb))
        cfg = ImageManipConfig()
        rr = RotatedRect()
        rr.center.x = (bb.xmin + bb.xmax) / 2
        rr.center.y = (bb.ymin + bb.ymax) / 2
        rr.size.width = bb.xmax - bb.xmin
        rr.size.height = bb.ymax - bb.ymin
        rr.angle = r # Rotate the rect in opposite direction
        # True = coordinates are normalized (0..1)
        cfg.setCropRotatedRect(rr, True)
        cfg.setResize(112, 112)
        cfg.setKeepAspectRatio(True)

        node.io['manip2_cfg'].send(cfg)
        node.io['manip2_img'].send(img)

The following is the error I got:

francis@raspberrypi:~/Desktop/learningOAK-D-Lite $ /bin/python /home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py
Creating pipeline...
Creating Color Camera...
Creating Face Detection Neural Network...
Creating Head pose estimation NN
Creating face recognition ImageManip/NN
[18443010F1AECE1200] [1.1.1] [2.030] [Script(6)] [critical] KeyError: ('frame',)

At:
  <script>(18): <module>

Traceback (most recent call last):
  File "/home/francis/Desktop/learningOAK-D-Lite/gen2-face-recognition/main.py", line 215, in <module>
    if q.has():
RuntimeError: Communication exception - possible device error/misconfiguration. Original message 'Couldn't read data from stream: 'color' (X_LINK_ERROR)'

I expected that some of the warning statements would show up in the terminal before the [critical] KeyError: ('frame',) would come out. However, that is not the case. It appears that the [critical] KeyError: ('frame',) occurred before the script.py code even ran. Is that possible? What else can I try to see if I can resolve the issue?

Thanks, Francis.

[Erol444]

Hi @FrancisTse8 , No, in general keyerror happens when you try to get value from dictionary, but key doesn't exist. That's why we also check for it inside in the code below. I have no idea how this could happen, seems like code that you shared isn't the same as what gets uploaded to the OAK. At least because it's a simple python logic it should be easy to debug.

if str(target_seq) in sync and "frame" in sync[str(target_seq)]:
        return sync[str(target_seq)]["frame"]

[FrancisTse8]

Hello @Erol444, You seem to be right about the script.py not being loaded onto the OAK device. I rename the script.py file to script_test.py. Running main.py produced the same error. There was no complain about not being able to find the script.py file. In the main.py, there is the code:

with open("script.py", "r") as f:
    script.setScript(f.read())

I thought main.py would read from the ./script.py file each time it is run. So, where else is main.py getting it's script.py input from?

[Erol444]

Hi @FrancisTse8 , Are you located (in terminal) in a different folder, and it's sourcing script.py from that folder instead? Just a guess, otherwise that's the question you would need to answer by yourself:) Thanks, Erik

[FrancisTse8]

Hello @Erol444, I created a copy of gen2-face-recognition folder and placed it on the desktop. When I ran main.py, it complained that it could not find the script.py file. I changed script_test.py back to script.py and main.py ran fine and I do not have the error anymore. So, the issue is with the correct script.py not being read. Guess I learned a lesson about checking if the correct script.py is being loaded. Will be closing this issue again. Thank you so much for your help and patience, Francis.