search

sclaflin / Plate-Minder

Monitor a video source for license plates and record them. Zero cloud dependencies.
MIT License
151 stars 22 forks source link

Crashes after a few days - memory leak? #22

Open jfarre20 opened 1 year ago

jfarre20 commented 1 year ago

Got this after a few days uptime, restarted the container and it did it again after a few days

Failed to process image: RuntimeError: memory access out of bounds

     at wasm://wasm/01728d26:wasm-function[124]:0xfa9c
     at wasm://wasm/01728d26:wasm-function[12338]:0x51848c
     at Module.dynCall_iiiii (/app/lib/opencv/opencv.js:30:8275315)
     at dynCall_iiiii_9 (eval at makeDynCaller (/app/lib/opencv/opencv.js:30:8226307), <anonymous>:4:12)
     at Mat.constructor_body (/app/lib/opencv/opencv.js:30:8235294)
     at Mat.<anonymous> (/app/lib/opencv/opencv.js:30:8229032)
     at new Mat (eval at createNamedFunction (/app/lib/opencv/opencv.js:30:8206372), <anonymous>:4:34)
     at RawImage.toMat (file:///app/lib/RawImage.js:174:15)
     at MaskImageFilter.next (file:///app/lib/MaskImageFilter.js:15:24)
     at PlateMinder.next (file:///app/lib/PlateMinder.js:122:18)

 Failed to process image: RuntimeError: memory access out of bounds
     at wasm://wasm/01728d26:wasm-function[124]:0xfa9c
     at wasm://wasm/01728d26:wasm-function[12338]:0x51848c
     at Module.dynCall_iiiii (/app/lib/opencv/opencv.js:30:8275315)
     at dynCall_iiiii_9 (eval at makeDynCaller (/app/lib/opencv/opencv.js:30:8226307), <anonymous>:4:12)
     at Mat.constructor_body (/app/lib/opencv/opencv.js:30:8235294)
     at Mat.<anonymous> (/app/lib/opencv/opencv.js:30:8229032)
     at new Mat (eval at createNamedFunction (/app/lib/opencv/opencv.js:30:8206372), <anonymous>:4:34)
     at RawImage.toMat (file:///app/lib/RawImage.js:174:15)
     at MaskImageFilter.next (file:///app/lib/MaskImageFilter.js:15:24)
     at PlateMinder.next (file:///app/lib/PlateMinder.js:122:18)

Error would repeat and spam the console

sclaflin commented 1 year ago

This error has an open issue with OpenCV.js: https://github.com/opencv/opencv/issues/19397

Though I'm not convinced it isn't something I've done. Are you able to check memory usage when it first starts, maybe a day later, and finally when the error crops up? Maybe there's a memory leak somewhere.

jfarre20 commented 1 year ago

This error has an open issue with OpenCV.js: opencv/opencv#19397

Maybe there's a memory leak somewhere.

there's very much a memory leak, it starts at around 4g used and after a day it's around 8g

I set the container to cap memory to 6.5 gigs, and this seems to restart the container once it goes over that

I have two 4K cameras H265, 15 fps. I'm not sure if the memory leak is coming from FFmpeg.

sclaflin commented 1 year ago

Can you post your (sanitized) config here? I'd like to try and replicate the issue.

jfarre20 commented 1 year ago 
sources:
  - type: rtsp
    name: Cam2
    captureInterval: 1
    preInputArgs: []
    preOutputArgs: []
    alwaysRestart: true
    url: rtsp://UN:PW@10.8.11.6/cam/realmonitor?channel=28&subtype=0
  - type: rtsp
    name: Cam3
    captureInterval: 1
    preInputArgs: []
    preOutputArgs: []
    alwaysRestart: true
    url: rtsp://UN:PW@10.8.11.6/cam/realmonitor?channel=30&subtype=0
filters:
  - type: mask
    debug: false
    shapes:
      - 0,0,700,0,3840,0,3840,1460
      - 0,1960,0,2160,560,2160,560,1960
  - type: motion
    debug: false
openALPR:
  url: http://open-alpr-http-wrapper:3000/detect
  country_code: us
  pattern: nc
recorders:
  - type: mqtt
    url: mqtt://homeassistant.DOMAIN.local:1883
    baseTopic: plate-minder
    mqttOptions:
      username: mqttunhere
      password: mqttpwhere
    hassDiscovery:
      enable: true
      discoveryPrefix: homeassistant
  - type: file
    pattern: ./images/{{DATE}}/{{SOURCE}}/{{TIME}}_{{PLATE}}.jpeg
    retainDays: 30
  - type: sqlite
restService:
  enable: true
  port: 4000
jfarre20 commented 1 year ago

also the docker compose

version: "3.9"
services:
  plate-minder:
    container_name: plate-minder
    restart: unless-stopped
    image: sclaflin/plate-minder:latest
    deploy:
      resources:
        limits:
          memory: 6.5g
    ports:
      - 4000:4000
    volumes:
      # Set's the docker container to the host container local time
      - /etc/localtime:/etc/localtime:ro
      - /data/plateminder/data:/app/data
      - /data/plateminder/config.yaml:/app/config.yaml
  open-alpr-http-wrapper:
    container_name: open-alpr-http-wrapper
    restart: unless-stopped
    image: sclaflin/open-alpr-http-wrapper:latest
  # This service is not required, but exists to help with configuration. Once 
  # Plate-Minder has been configured, feel free to disable this service.
  plate-minder-web:
    container_name: plate-minder-web
    image: sclaflin/plate-minder-web:latest
    restart: unless-stopped
    # The default configuration assumes docker is running on the same machine 
    # you're viewing the web UI with. If you're accessing the service from a
    # different computer, you should set the PLATE_MINDER_URL to whatever host 
    # & port Plate-Minder's RESTful service is accessible from.
    environment:
      - PLATE_MINDER_URL=http://10.8.8.8:4000
    ports:
      - 8100:80
crookedmosquito commented 8 months ago

Seeing same issue:

Failed to process image: RuntimeError: memory access out of bounds at wasm://wasm/01728d26:wasm-function[124]:0xfa9c at wasm://wasm/01728d26:wasm-function[12338]:0x51848c at Module.dynCall_iiiii (/app/lib/opencv/opencv.js:30:8275315) at dynCall_iiiii_9 (eval at makeDynCaller (/app/lib/opencv/opencv.js:30:8226307), :4:12) at Mat.constructor_body (/app/lib/opencv/opencv.js:30:8235294) at Mat. (/app/lib/opencv/opencv.js:30:8229032) at new Mat (eval at createNamedFunction (/app/lib/opencv/opencv.js:30:8206372), :4:34) at RawImage.toMat (file:///app/lib/RawImage.js:174:15) at MaskImageFilter.next (file:///app/lib/MaskImageFilter.js:15:24) at PlateMinder.next (file:///app/lib/PlateMinder.js:122:18) Failed to process image: RuntimeError: memory access out of bounds at wasm://wasm/01728d26:wasm-function[124]:0xfa9c at wasm://wasm/01728d26:wasm-function[12338]:0x51848c at Module.dynCall_iiiii (/app/lib/opencv/opencv.js:30:8275315) at dynCall_iiiii_9 (eval at makeDynCaller (/app/lib/opencv/opencv.js:30:8226307), :4:12) at Mat.constructor_body (/app/lib/opencv/opencv.js:30:8235294) at Mat. (/app/lib/opencv/opencv.js:30:8229032) at new Mat (eval at createNamedFunction (/app/lib/opencv/opencv.js:30:8206372), :4:34) at RawImage.toMat (file:///app/lib/RawImage.js:174:15) at MaskImageFilter.next (file:///app/lib/MaskImageFilter.js:15:24) at PlateMinder.next (file:///app/lib/PlateMinder.js:122:18)