Ability to customize order of output for custom models

alec-groff commented 8 months ago

Describe the problem you are having

ssd-mobilenet-v2 (tfl2) object detection output does not match the output required for EdgeTpuTfl detect_raw function. The only accepted output order is [boxes, class_ids, scores, count], so I am getting the below error for my model that outputs [scores, boxes, count, class_ids]. My model works locally when using the following pycoral script to test: https://github.com/google-coral/pycoral/blob/master/examples/detect_image.py.

Version

0.12.1-367d724

Frigate config file

detectors:
  coral:
    type: edgetpu
    device: usb

model:
  path: /tflite/detect_quant.tflite
  width: 320
  height: 320
  # input_tensor: nhwc
  # input_pixel_format: bgr
  labelmap_path: /tflite/labelmap.txt

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "256mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb # passes the USB Coral, needs to be modified for other versions
      - /dev/apex_0:/dev/apex_0 # passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
      - /dev/dri/renderD128 # for intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ./config/config.yml:/config/config.yml
      - ./storage:/media/frigate
      - ./models/tflite:/tflite # <-- tflite edgeTPU Google Coral
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"

Relevant log output

frigate  | 2024-01-24 16:35:02.971151445  Process detector:coral:
frigate  | 2024-01-24 16:35:02.971650574  Traceback (most recent call last):
frigate  | 2024-01-24 16:35:02.971652692    File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
frigate  | 2024-01-24 16:35:02.971653765      self.run()
frigate  | 2024-01-24 16:35:02.971654953    File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
frigate  | 2024-01-24 16:35:02.971655955      self._target(*self._args, **self._kwargs)
frigate  | 2024-01-24 16:35:02.971656982    File "/opt/frigate/frigate/object_detection.py", line 121, in run_detector
frigate  | 2024-01-24 16:35:02.971657991      detections = object_detector.detect_raw(input_frame)
frigate  | 2024-01-24 16:35:02.971659054    File "/opt/frigate/frigate/object_detection.py", line 71, in detect_raw
frigate  | 2024-01-24 16:35:02.971660203      return self.detect_api.detect_raw(tensor_input=tensor_input)
frigate  | 2024-01-24 16:35:02.971662224    File "/opt/frigate/frigate/detectors/plugins/edgetpu_tfl.py", line 61, in detect_raw
frigate  | 2024-01-24 16:35:02.971663697      count = int(
frigate  | 2024-01-24 16:35:02.971665351  TypeError: only size-1 arrays can be converted to Python scalars

Operating system

Debian

Install method

Docker Compose

Coral version

USB

Any other information that may be helpful

No response

NickM-27 commented 8 months ago

This is working as expected currently. Are you hoping to request a feature where this is customizable in the config?

alec-groff commented 8 months ago

Yes sorry that was my intention here. Not a bug, just requesting a feature. I noticed others in Issue #3437 had the same problem that I believe can be fixed with a small feature.

blakeblackshear / frigate