Motion Mask not working (mog2 and background_subtractor)

[Edgar-ZP]

Hi Hi,

first of all: thanks for this great project !

This weekend I updated my setup (finally) to the latest 2.x version. And while the config is completely different, it reads very nice.

Only the big problem I face is that I can´t get the motion-mask to work anymore. And with a big road just on the edge of my camera I get a lot of triggers :-(

I tried all kinds of different configs:

• First of all, on which resolution do I have to draw the motion mask ? I tried both, the resolution in the ffmpeg config as in the mog2/background_subtractor. Both times if I draw the mask on the video stream it shows just perfect.
• Also did i tried to make the mask just some pixels smaller, so i would not go over the borders.
• I tried first mog2 and later also the background_subtractor.
• I tried to enable the mask just on one camera...
• Checked the logs every time, also in debug, but no error or warning...

I there a bug in the latest version (v2.2.0) or do I miss a config setting ?

Kind regards !

This is my complete config:

ffmpeg:
  camera:
    camera_31_frontdoor:
      path: /cam/stream1
      port: 554
      host: 172.20.8.31
      name: Home Frontdoor
      mjpeg_streams:
        frontdoor_default:
          width: 800
          height: 600
          draw_objects: true
          draw_motion: true
          draw_motion_mask: false
          draw_object_mask: false
          draw_zones: false
          rotate: 0
          mirror: false
      stream_format: rtsp
      protocol: rtsp
      width: 1600
      height: 1200
      fps: 6
      #input_args:
      #hwaccel_args:
      audio_codec: aac
      rtsp_transport: tcp
      #video_filters:
      pix_fmt: nv12
      frame_timeout: 30
      username: !secret camera_31_frontdoor_user
      password: !secret camera_31_frontdoor_pass
      #global_args:
      #substream: <<<<<
      ffmpeg_loglevel: error
      #ffmpeg_recoverable_errors:
      ffprobe_loglevel: error
      recorder:
        lookback: 20
        idle_timeout: 20
        retain: 31
        #folder:
        #filename_pattern:
        #extension:
        thumbnail:
          save_to_disk: true
          #filename_pattern:
        #hwaccel_args:
        #codec:
        #audio_codec:
        #video_filters:
        #audio_filters:
        #segments_folder:
    camera_33_driveway:
      path: /cam/stream1
      port: 554
      host: 172.20.8.33
      name: Home Driveway
      mjpeg_streams:
        driveway_default:
          width: 854
          height: 480
          draw_objects: true
          draw_motion: true
          draw_motion_mask: false
          draw_object_mask: false
          draw_zones: false
          rotate: 0
          mirror: false
      stream_format: rtsp
      protocol: rtsp
      width: 2560
      height: 1440
      fps: 6
      #fps: 25
      #input_args:
      #hwaccel_args:
      #audio_codec:
      rtsp_transport: tcp
      #video_filters:
      pix_fmt: nv12
      frame_timeout: 30
      username: !secret camera_33_driveway_user
      password: !secret camera_33_driveway_pass
      #global_args:
      #substream: <<<<<
      ffmpeg_loglevel: error
      #ffmpeg_recoverable_errors:
      ffprobe_loglevel: error
      recorder:
        lookback: 20
        idle_timeout: 20
        retain: 31
        #folder:
        #filename_pattern:
        #extension:
        thumbnail:
          save_to_disk: true
          #filename_pattern:
        #hwaccel_args:
        #codec:
        #audio_codec:
        #video_filters:
        #audio_filters:
        #segments_folder:
    camera_34_shack:
      path: /cam/stream1
      port: 554
      host: 172.20.8.34
      name: Home Shack
      mjpeg_streams:
        shack_default:
          width: 854
          height: 480
          draw_objects: true
          draw_motion: true
          draw_motion_mask: false
          draw_object_mask: false
          draw_zones: false
          rotate: 0
          mirror: false
      stream_format: rtsp
      protocol: rtsp
      width: 2560
      height: 1440
      fps: 6
      #fps: 25
      #input_args:
      #hwaccel_args:
      #audio_codec:
      rtsp_transport: tcp
      #video_filters:
      pix_fmt: nv12
      frame_timeout: 30
      username: !secret camera_34_shack_user
      password: !secret camera_34_shack_pass
      #global_args:
      #substream: <<<<<
      ffmpeg_loglevel: error
      #ffmpeg_recoverable_errors:
      ffprobe_loglevel: error
      recorder:
        lookback: 20
        idle_timeout: 20
        retain: 31
        #folder:
        #filename_pattern:
        #extension:
        thumbnail:
          save_to_disk: true
          #filename_pattern:
        #hwaccel_args:
        #codec:
        #audio_codec:
        #video_filters:
        #audio_filters:
        #segments_folder:

background_subtractor:
  motion_detector:
    cameras:
      camera_31_frontdoor:
        trigger_recorder: false
        recorder_keepalive: true
        #max_recorder_keepalive
        fps: 4 
        area: 0.01
        # Orgineel is 1600x1200 (=1.92) 800x600 (0.48)
        width: 800
        height: 600
        #mask:
          #- coordinates:
          #    - x: 0
          #      y: 0
          #    - x: 0
          #      y: 330
          #    - x: 523
          #      y: 330
          #    - x: 523
          #      y: 233
          #    - x: 675
          #      y: 118
          #    - x: 799
          #      y: 65
          #    - x: 799
          #      y: 0
        threshold: 32
        ###history: 500
        ###detect_shadows: false
        ###learning_rate: 0.01
      camera_33_driveway:
        trigger_recorder: false
        recorder_keepalive: true
        #max_recorder_keepalive
        fps: 4
        area: 0.01
        # Orgineel is 2560x1440 (=3.68) 854x480 (0.41)
        width: 854
        height: 480
        mask:
          #- coordinates:
          #    - x: 2559
          #      y: 1
          #    - x: 2559
          #      y: 1439
          #    - x: 2310
          #      y: 1439
          #    - x: 2310
          #      y: 510
          #    - x: 1935
          #      y: 165
          #    - x: 1260
          #      y: 60
          #    - x: 1260
          #      y: 1
          - coordinates:
              - x: 853
                y: 1
              - x: 853
                y: 479
              - x: 770
                y: 479
              - x: 770
                y: 170
              - x: 645
                y: 55
              - x: 420
                y: 20
              - x: 420
                y: 1
        threshold: 32
        ###history: 500
        ###detect_shadows: false
        ###learning_rate: 0.01
      camera_34_shack:
        trigger_recorder: false
        recorder_keepalive: true
        #max_recorder_keepalive
        fps: 4
        area: 0.01
        # Orgineel is 2560x1440 (=3.68) 854x480 (0.41)
        width: 854
        height: 480
        # mask:
        threshold: 32
        ###history: 500
        ###detect_shadows: false
        ###learning_rate: 0.01

darknet:
  object_detector:
    cameras:
      camera_31_frontdoor:
        fps: 1
        scan_on_motion_only: true
        labels:
          - label: person
            confidence: 0.7
            #height_min:
            #height_max:
            #width_min:
            #width_max:
            trigger_recorder: true
            require_motion: true
          - label: bicycle
            confidence: 0.7
            trigger_recorder: true
            require_motion: true
          - label: car
            confidence: 0.7
            trigger_recorder: true
            require_motion: true
        max_frame_age: 10
        log_all_objects: false
        mask:
          - coordinates:
              - x: 60
                y: 610
              - x: 400
                y: 610
              - x: 400
                y: 790
              - x: 60
                y: 790
        #zones: <<<
      camera_33_driveway:
        fps: 1
        scan_on_motion_only: true
        labels:
          - label: person
            confidence: 0.7
            trigger_recorder: true
            require_motion: true
          - label: bicycle
            confidence: 0.7
            trigger_recorder: true
            require_motion: true
          - label: car
            confidence: 0.7
            trigger_recorder: true
            require_motion: true
      camera_34_shack:
        fps: 1
        scan_on_motion_only: true
        labels:
          - label: person
            confidence: 0.7
            trigger_recorder: true
            require_motion: true
          - label: bicycle
            confidence: 0.7
            trigger_recorder: true
            require_motion: true
          - label: car
            confidence: 0.7
            trigger_recorder: false
            require_motion: true
        max_frame_age: 5
        log_all_objects: false
        #mask: <<<
        #zones: <<<
    model_path: /detectors/models/darknet/yolov7-tiny.weights
    model_config: /detectors/models/darknet/yolov7-tiny.cfg
    label_path: /detectors/models/darknet/coco.names
    #suppression:
    #dnn_backend:
    #dnn_target:
    #half_precision:

nvr:
  camera_31_frontdoor:
  camera_33_driveway:
  camera_34_shack:

webserver:
  port: 8888
  debug: false

logger:
  default_level: info
  #logs:
    #viseron.components.edgetpu: debug
  cameras:
    #camera_31_frontdoor: debug
    #camera_33_driveway: debug

mqtt:
#secret :-)

[roflcoopter]

Hi! Glad you've been enjoying Viseron :)

Nothing has changed in 2.X whn it comes to masking, so it should still be working. The mask should be configured using coordinates of the cameras resolution.

If you check the live view and add ?draw_motion_mask=1 at the end, does it look the way you expect? The full URL looks like this: http://<server ip>:<port>/<camera identifier>/mjpeg-stream?draw_motion_mask=1

[Edgar-ZP]

When I restart Viseron with the same config, but only the other mask (the one with the # in front of in my config snippet) of camera_33_driveway. I get indeed the mask the way I want on http://x.x.x.x:8888/camera_33_driveway/mjpeg-stream?draw_motion_mask=1.

For the docs it would be nice if you could add a comment that the mask is about the original (ffmpeg component / camera) resolution.

[Edgar-ZP]

I have tried some other config and even other locations of the motion mask, but it seems to me the mask is never applied. But it is all on try and error. Do you have some idea I could do to debug this and generate more log data ?

[roflcoopter]

Hmm, if you check http://x.x.x.x:8888/camera_33_driveway/mjpeg-stream?draw_motion_mask=1&draw_motion=1 do the squiggly lines (representing motion) intersect with the mask?

In my tests the mask is applied correctly

[Edgar-ZP]

Yes I see the outlines of the motion within the mask. It's like the mask isn't applied at all, but the mask is drawn perfect on the video. I also tried a simple box and different places, but without any luck so I am out-of-ideas to debug.

And because with every change I have to go outside and run around it costs a lot of time...

[roflcoopter]

Will run some tests and get report back if i can manage to reproduce or not, thanks for your patience

[freisei]

Yes I see the outlines of the motion within the mask.

I think you have to negate your mask. Motion is only triggered outside the mask, not inside it.

[roflcoopter]

That is correct, motion is masked within the region that you defined.

I have not had the time to run tests yet. Sorry about that

[Edgar-ZP]

Yesterday night I was playing around, again, with the config and it seems I have one Motion Mask working (camera 31). It is not the camera i wanted to, but it is progress. But i needed to "shrink" the motion mask far within the picture.

The second camera (32) still does not work with the motion mask.

mog2:
  motion_detector:
    cameras:
      camera_31_frontdoor:
        trigger_recorder: false
        recorder_keepalive: true
        #max_recorder_keepalive
        fps: 4 
        area: 0.007
        width: 800
        height: 600
        mask:
          - coordinates:
              - x: 10
                y: 10
              - x: 10
                y: 660
              - x: 1048
                y: 660
              - x: 1048
                y: 466
              - x: 1350
                y: 238
              - x: 1590
                y: 130
              - x: 1590
                y: 10
        threshold: 20
        history: 500
        detect_shadows: false
        learning_rate: 0.01
      camera_33_driveway:
        trigger_recorder: false
        recorder_keepalive: true
        #max_recorder_keepalive
        fps: 4
        area: 0.007
        width: 854
        height: 480
        mask:
          - coordinates:
              - x: 2550
                y: 10
              - x: 2550
                y: 550
              - x: 2240
                y: 502
              - x: 1935
                y: 165
              - x: 1260
                y: 70
              - x: 1260
                y: 10
        threshold: 20
        history: 500
        detect_shadows: false
        learning_rate: 0.01

[Edgar-ZP]

Hmmm and it is broken again... Could it be the mog2 motion-mask crashes or something without logs ? It is of course strange the motion detection keeps working. :exploding_head:

[roflcoopter]

That should not be possible no, the mask is applied to the image in the same thread, so if the masking would crash it would stop detecting motion as well.

I've had limited time the last few weeks so i have not been able to dig further just yet. Im thinking about adding a new view to the frontend which would allow you to see what Viseron "sees", maybe that could give some clues.

[roflcoopter]

Could you navigate to /<CAMERA_IDENTIFIER>/mjpeg-stream?draw_motion_mask=1&draw_motion=1 and show me what it looks like when you say it stops working?

I've been testing and from what i can observe it is working. There might be an issue with the area being misscalculated if the motion contour entirely surrounds the mask, gonna do some extra testing on that

[Edgar-ZP]

I cropped the picture of privacy reasons, i do not like it so public. But you can see the problem, the headlights of traffic on the road triggers motion.

There is a little room between the mask and the border of the picture, this was because I thought this would fix the problem. But as you can see it didn´t...

[roflcoopter]

Interesting, thanks for providing the images. Will copy your config and run some tests

[roflcoopter]

I cant for the life of me reproduce this. I am thinking it could be because i am referencing a global frame instead of copying it. I just pushed that change to dev, will take a few minutes for it to finish the build.

Can you try that and see if it helps?

[zylan1]

Hey. I seem to be having the exact issue with an object mask. The mask is drawn properly, but not applied. I've got a stupid tree in the area that waves like a person. Anything I can do to help?

darknet:
  object_detector:
    cameras:
      camera_1:  # Attach detector to the configured camera_1 above
        fps: 1
        scan_on_motion_only: false  # Scan for objects even when there is no motion
        labels:
          - label: person
            confidence: 0.90
            trigger_recorder: true
            height_min: 0.08
        mask:
        - coordinates:
          - x: 5
            y: 0
          - x: 1274
            y: 3
          - x: 12
            y: 709

[Edgar-ZP]

I just upgraded to version v2.3.1 but this problem still exists. While the motion mask is perfect drawn on the mjpeg-stream, motions inside the mask are still recognized and trigger a motion.

[roflcoopter]

Will need to take a deeper look. I did some tests a few weeks back but i could not reproduce. Will need to investigate deeper.

Working hard on #619 atm so it might take some time, i have not forgotten about it tho!

[Edgar-ZP]

:+1: When you have an idea of build i can test, just let me know. And thanks for sharing this nice project !