Question: How to Limit Detection Region

[DeadEnded]

Hello! I am playing around with using Node-Red to pass the payload to the API for scans. So far it has been working very well, but I need to cut off the left 10% of the detection region to prevent a false positive from a tree. I know how to do this from Home Assistant, but was hoping there was a way to pass it in the payload from Node-Red.

This is my current testing payload:

msg.payload = {
    "detector_name": "tensorflow",
    "data": msg.image.toString('base64'),
    "detect": {"*": 50},
}
return msg;

msg.image is where I have a snapshot from a camera - and this works perfectly.

I have tried adding a plethora of things to get the DetectionRegion that I found in the rpc.pb.go and rpc.proto files. I have tried DetectionRegions, DetectionRegion, Regions, Region - as new tags after detect, and also as tags inside the detect tag. I'm a novice and so haven't figured out from the source exactly where it is suppose to go... :-)

Can you please assist and let me know where in the payload I can put this to get the result of "left":0.1 to prevent the left 10% of the image from being in the detection range?

Or am I chasing a ghost and this isn't even supported in the API call?

Thanks very much! DeadEnd

forgot to add... depending on what I do either the detection works without the limited region, or I get an error like this:

json: cannot unmarshal object into Go struct field DetectRequest.regions of type []*odrpc.DetectRegion

or

json: cannot unmarshal object into Go struct field DetectRequest.detect of type float32

I assume the first error I'm closer as I get an error about the regions - the second I think my yaml is not as correct. Here is an example of what works, but doesn't seem to effect the detection region:

msg.payload = {
    "detector_name": "tensorflow",
    "data": msg.image.toString('base64'),
    "detect": {"*": 50},
    "region": {"left": 0.1}
}
return msg;

[snowzach]

Hey yeah, it's not supported yet. I started working on it but it's not completed yet.

[snowzach]

Okay, I just pushed a change adding support for it.. Give a shot and let me know.

[DeadEnded]

Awesome! That was fast! I tested it out per the docs... didn't work initially - there is a typo in the readme.

  "regions": [
    {
      "top": 0,
      "left": 0,
      "bottom": 1,
      "right": 1,
      "labels": {
        "person": 50,
        "*": 90
      },
      "covers": true
    }
  ]
}

labels should be detect:

  "regions": [
    {
      "top": 0,
      "left": 0,
      "bottom": 1,
      "right": 1,
      "detect": {
        "person": 50,
        "*": 90
      },
      "covers": true
    }
  ]
}

I played around with it until I figure this out and now it appears to be working (I have a car in the driveway and can get it to not see it). Interestingly the detector log in the container still shows 2 detections, but they aren't in the payload. So the detection appears to still happen on the full image, but the payload is restricted by the region... makes perfect sense since you can have multiple regions... one scan, then break up the payload by region.

Anyway - looks like it works. Just have to change the docs to say detect instead of labels. Labels are the returned tag I believe.

Cheers! DeadEnd

[edasque]

@DeadEnded would you share your node-red flow?

[DeadEnded]

The entire flow is somewhat more complicated and contains other things unrelated. Here are the steps relevant getting an image, and processing it:

[
    {
        "id": "53e69603.eec538",
        "type": "http request",
        "z": "1f63d8b5.dcfe47",
        "name": "File",
        "method": "GET",
        "ret": "bin",
        "paytoqs": "ignore",
        "url": "<CAMERA SNAPSHOT URL>",
        "tls": "",
        "persist": true,
        "proxy": "",
        "authType": "",
        "x": 440,
        "y": 156,
        "wires": [
            [
                "1cd226a1.91ca59"
            ]
        ]
    },
    {
        "id": "1cd226a1.91ca59",
        "type": "change",
        "z": "1f63d8b5.dcfe47",
        "name": "Move (preserve image)",
        "rules": [
            {
                "t": "move",
                "p": "payload",
                "pt": "msg",
                "to": "image",
                "tot": "msg"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 630,
        "y": 156,
        "wires": [
            [
                "bd9d07e5.9584a8"
            ]
        ]
    },
    {
        "id": "bd9d07e5.9584a8",
        "type": "function",
        "z": "1f63d8b5.dcfe47",
        "name": "Build Payload",
        "func": "msg.payload = {\n    \"detector_name\": \"tensorflow\",\n    \"data\": msg.image.toString('base64'),\n//    \"detect\": {\"*\": 50},\n    \"regions\": [\n    {\n      \"top\": 0,\n      \"left\": 0.15,\n      \"bottom\": 1,\n      \"right\": 1,\n      \"detect\": {\n        \"person\": 90,\n      },\n      \"covers\": true\n    }\n    ]\n}\n//msg.headers = {\n//    \"Content-Type\": \"application/json\"\n//}\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "x": 834,
        "y": 156,
        "wires": [
            [
                "d086fce3.c2935"
            ]
        ]
    },
    {
        "id": "d086fce3.c2935",
        "type": "http request",
        "z": "1f63d8b5.dcfe47",
        "name": "Detector",
        "method": "POST",
        "ret": "obj",
        "paytoqs": "ignore",
        "url": "http://<DOODS IP:port>/detect",
        "tls": "",
        "persist": true,
        "proxy": "",
        "authType": "",
        "x": 996,
        "y": 156,
        "wires": [
            []
        ]
    }
]

[edasque]

Thank you, that's exciting, I'll try it out.