search

kerberos-io / agent

An open and scalable video surveillance system for anyone making this world a better and more peaceful place.
https://kerberos.io
MIT License
640 stars 78 forks source link

Problem creating a filemkdir open .data/snapshots/.test permission denied #62

Closed instantdreams closed 1 year ago

instantdreams commented 1 year ago

When I start my containers using Docker Compose I receive the following errors in the log:

$ docker compose --file /srv/kerberos-agent/docker-compose.yml logs
ka-garage-south  | {"level":"fatal","msg":"Problem creating a file: open ./data/snapshots/.test: permission denied","time":"2022-12-19T20:37:39+01:00"}
ka-garage-south  | {"level":"fatal","msg":"Problem creating a file: open ./data/snapshots/.test: permission denied","time":"2022-12-19T20:37:42+01:00"}
ka-garage-north     | {"level":"fatal","msg":"Problem creating a file: open ./data/snapshots/.test: permission denied","time":"2022-12-19T20:37:39+01:00"}
ka-garage-north     | {"level":"fatal","msg":"Problem creating a file: open ./data/snapshots/.test: permission denied","time":"2022-12-19T20:37:42+01:00"}

Here is my sanitised docker-compose.yml configuration:

$ docker compose --file /srv/kerberos-agent/docker-compose.yml config
name: kerberos-agent
services:
  ka-garage-north:
    container_name: ka-garage-north
    environment:
      AGENT_AUTO_CLEAN: "true"
      AGENT_AUTO_CLEAN_MAX_SIZE: "102400"
      AGENT_CAPTURE_CONTINUOUS: "false"
      AGENT_CAPTURE_FRAGMENTED: "false"
      AGENT_CAPTURE_FRAGMENTED_DURATION: "8"
      AGENT_CAPTURE_IPCAMERA_RTSP: rtsp://[user]:[pass]@192.168.1.84:554/cam/realmonitor?channel=1&subtype=0
      AGENT_CAPTURE_IPCAMERA_SUB_RTSP: rtsp://[user]:[pass]@192.168.1.84:554/cam/realmonitor?channel=1&subtype=1
      AGENT_CAPTURE_MAXLENGTH: "600"
      AGENT_CAPTURE_PIXEL_CHANGE: "150"
      AGENT_CAPTURE_POSTRECORDING: "40"
      AGENT_CAPTURE_PRERECORDING: "20"
      AGENT_KEY: ""
      AGENT_MQTT_PASSWORD: ""
      AGENT_MQTT_URI: tcp://195.168.1.95:1883
      AGENT_MQTT_USERNAME: ""
      AGENT_NAME: garage-north
      AGENT_OFFLINE: "false"
      AGENT_PASSWORD: [password]
      AGENT_TIMEZONE: America/Edmonton
      AGENT_USERNAME: admin
    hostname: [hostname]
    image: kerberos/agent:latest
    networks:
      default: null
    ports:
    - mode: ingress
      target: 80
      published: "8083"
      protocol: tcp
    restart: unless-stopped
    user: 1000:1000
    volumes:
    - type: bind
      source: /srv/kerberos-agent/garage-north/config
      target: /home/agent/data/config
      bind:
        create_host_path: true
    - type: bind
      source: /media/nvr/kerberos/garage-north/recordings
      target: /home/agent/data/recordings
      bind:
        create_host_path: true
  ka-garage-south:
    container_name: ka-garage-south
    environment:
      AGENT_AUTO_CLEAN: "true"
      AGENT_AUTO_CLEAN_MAX_SIZE: "102400"
      AGENT_CAPTURE_CONTINUOUS: "false"
      AGENT_CAPTURE_FRAGMENTED: "false"
      AGENT_CAPTURE_FRAGMENTED_DURATION: "8"
      AGENT_CAPTURE_IPCAMERA_RTSP: rtsp://[user]:[pass]@192.168.1.83:554/cam/realmonitor?channel=1&subtype=0
      AGENT_CAPTURE_IPCAMERA_SUB_RTSP: rtsp://[user]:[pass]@192.168.1.83:554/cam/realmonitor?channel=1&subtype=1
      AGENT_CAPTURE_MAXLENGTH: "600"
      AGENT_CAPTURE_PIXEL_CHANGE: "150"
      AGENT_CAPTURE_POSTRECORDING: "40"
      AGENT_CAPTURE_PRERECORDING: "20"
      AGENT_KEY: ""
      AGENT_MQTT_PASSWORD: ""
      AGENT_MQTT_URI: tcp://195.168.1.95:1883
      AGENT_MQTT_USERNAME: ""
      AGENT_NAME: garage-south
      AGENT_OFFLINE: "false"
      AGENT_PASSWORD: [password]
      AGENT_TIMEZONE: America/Edmonton
      AGENT_USERNAME: admin
    hostname: [hostname]
    image: kerberos/agent:latest
    networks:
      default: null
    ports:
    - mode: ingress
      target: 80
      published: "8082"
      protocol: tcp
    restart: unless-stopped
    user: 1000:1000
    volumes:
    - type: bind
      source: /srv/kerberos-agent/garage-south/config
      target: /home/agent/data/config
      bind:
        create_host_path: true
    - type: bind
      source: /media/nvr/kerberos/garage-south/recordings
      target: /home/agent/data/recordings
      bind:
        create_host_path: true
networks:
  default:
    name: kerberos-agent_default

The mount points for these two services are as follows:

$ docker container inspect ka-garage-south -f '{{range .Mounts}}{{.Type}}:{{.Source}}:{{.Destination}}{{println}}{{ end }}'
bind:/srv/kerberos-agent/garage-south/config:/home/agent/data/config
bind:/media/nvr/kerberos/garage-south/recordings:/home/agent/data/recordings

$ docker container inspect ka-garage-east -f '{{range .Mounts}}{{.Type}}:{{.Source}}:{{.Destination}}{{println}}{{ end }}'
bind:/srv/kerberos-agent/garage-east/config:/home/agent/data/config
bind:/media/nvr/kerberos/garage-east/recordings:/home/agent/data/recordings

I am running the services under User idsvc:

$ id $user
uid=1000(idsvc) gid=1000(idsvc) groups=1000(idsvc),4(adm),20(dialout),24(cdrom),27(sudo),29(audio),44(video),46(plugdev),60(games),100(users),104(input),106(render),108(netdev),995(docker),997(gpio),998(i2c),999(spi)

Before starting the containers I ran the following commands:

$ mkdir /srv/kerberos-agent/garage-north /srv/kerberos-agent/garage-north/config
$ mkdir /srv/kerberos-agent/garage-south /srv/kerberos-agent/garage-south/config
$ sudo chown idsvc:idsvc /srv/kerberos-agent/ -R
$ sudo chmod 777 /srv/kerberos-agent/ -R

$ mkdir /media/nvr/kerberos/garage-north
$ mkdir /media/nvr/kerberos/garage-south
$ sudo chown idsvc:idsvc /media/nvr/kerberos/ -R
$ sudo chmod 777 /media/nvr/kerberos -R

The file permissions for the /home/agent/data/config mount look like this:

$ ls -la /srv/kerberos-agent/garage-north/
total 12
drwxrwxrwx 3 idsvc idsvc 4096 Dec 19 11:32 .
drwxrwxrwx 7 idsvc idsvc 4096 Dec 19 12:30 ..
drwxrwxrwx 2 idsvc idsvc 4096 Dec 19 12:56 config
$ ls -la /srv/kerberos-agent/garage-north/config/
total 8
drwxrwxrwx 2 idsvc idsvc 4096 Dec 19 12:56 .
drwxrwxrwx 3 idsvc idsvc 4096 Dec 19 11:32 ..
$ ls -la /srv/kerberos-agent/garage-south/
total 12
drwxrwxrwx 3 idsvc idsvc 4096 Dec 19 11:32 .
drwxrwxrwx 7 idsvc idsvc 4096 Dec 19 12:30 ..
drwxrwxrwx 2 idsvc idsvc 4096 Dec 19 12:56 config
$ ls -la /srv/kerberos-agent/garage-south/config/
total 8
drwxrwxrwx 2 idsvc idsvc 4096 Dec 19 12:56 .
drwxrwxrwx 3 idsvc idsvc 4096 Dec 19 11:32 ..

The file permissions for the /home/agent/data/recordings mount look like this:

$ ls -la /media/nvr/kerberos/garage-north/
total 16
drwxrwxrwx 4 idsvc idsvc 4096 Dec 19 12:37 .
drwxrwxrwx 7 idsvc idsvc 4096 Dec 19 11:32 ..
drwxrwxrwx 2 idsvc idsvc 4096 Dec 19 12:56 recordings
drwxrwxrwx 2 idsvc idsvc 4096 Dec 19 12:37 snapshots
$ ls -la /media/nvr/kerberos/garage-north/recordings/
total 8
drwxrwxrwx 2 idsvc idsvc 4096 Dec 19 12:56 .
drwxrwxrwx 4 idsvc idsvc 4096 Dec 19 12:37 ..

$ ls -la /media/nvr/kerberos/garage-south/
total 16
drwxrwxrwx 4 idsvc idsvc 4096 Dec 19 12:37 .
drwxrwxrwx 7 idsvc idsvc 4096 Dec 19 11:32 ..
drwxrwxrwx 2 idsvc idsvc 4096 Dec 19 12:56 recordings
drwxrwxrwx 2 idsvc idsvc 4096 Dec 19 12:37 snapshots
$ ls -la /media/nvr/kerberos/garage-south/recordings/
total 8
drwxrwxrwx 2 idsvc idsvc 4096 Dec 19 12:56 .
drwxrwxrwx 4 idsvc idsvc 4096 Dec 19 12:37 ..

My versions for reference:

$ docker --version
Docker version 20.10.21, build baeda1f
$ docker compose version
Docker Compose version v2.12.2

I have tried adding user: 1000:1000 to the services entries in the compose file, running the docker compose elevated, and I am stuck.

What incredibly obvious thing have I missed?

cedricve commented 1 year ago

hey @instantdreams , thanks for the incredible awesome logs ;) I'm having a look right now!

cedricve commented 1 year ago

We do set the USER to agent in the dockerfile. https://github.com/kerberos-io/agent/blob/master/Dockerfile#L162 That is the only thing, I'm wondering about.. Where as you are chowning.

sudo chown idsvc:idsvc /srv/kerberos-agent/ -R
instantdreams commented 1 year ago

If I could get my container to stay up I'd docker exec -it ka-garage-south bash then try id $user to see what PID and GID user agent was running as.

The fact that this isn't working either as a base up or with sudo makes me ponder.

instantdreams commented 1 year ago

Actually can you tell me the PID for agent and the GID for kerberosio ?

cedricve commented 1 year ago

So I've added another bind to your config and that did the trick @instantdreams 

  - type: bind
    source: .../kerberos/garage-north/snapshots
    target: /home/agent/data/snapshots
    bind:
      create_host_path: true

You also need to make sure you download a blank config in theconfig directory.

wget https://raw.githubusercontent.com/kerberos-io/agent/master/machinery/data/config/config.json -O agent/config/config.json
cedricve commented 1 year ago

I also noticed if you get rid of the user: 1000:1000, you can get rid of the config binding (as you override with environment variables).

name: kerberos-agent
services:
  ka-garage-north:
    container_name: ka-garage-north
    environment:
      AGENT_AUTO_CLEAN: "true"
      AGENT_AUTO_CLEAN_MAX_SIZE: "102400"
      AGENT_CAPTURE_CONTINUOUS: "false"
      AGENT_CAPTURE_FRAGMENTED: "false"
      AGENT_CAPTURE_FRAGMENTED_DURATION: "8"
      AGENT_CAPTURE_IPCAMERA_RTSP: rtsp://xxx:xxx@192.168.1.62/defaultPrimary?streamType=u
      AGENT_CAPTURE_IPCAMERA_SUB_RTSP: rtsp://xxx:xxx@192.168.1.62/defaultPrimary?streamType=u
      AGENT_CAPTURE_MAXLENGTH: "600"
      AGENT_CAPTURE_PIXEL_CHANGE: "150"
      AGENT_CAPTURE_POSTRECORDING: "40"
      AGENT_CAPTURE_PRERECORDING: "20"
      AGENT_KEY: ""
      AGENT_MQTT_PASSWORD: ""
      AGENT_MQTT_URI: tcp://195.168.1.95:1883
      AGENT_MQTT_USERNAME: ""
      AGENT_NAME: garage-north
      AGENT_OFFLINE: "false"
      AGENT_PASSWORD: root
      AGENT_TIMEZONE: America/Edmonton
      AGENT_USERNAME: admin
    hostname: ka-garage-north
    image: kerberos/agent:latest
    networks:
      default: null
    ports:
      - mode: ingress
        target: 80
        published: "8083"
        protocol: tcp
    restart: unless-stopped
    volumes:
      - type: bind
        source: /Users/xxx/Desktop/kerberos/garage-north/recordings
        target: /home/agent/data/recordings
        bind:
          create_host_path: true
      - type: bind
        source: /Users/xxx/Desktop/kerberos/garage-north/snapshots
        target: /home/agent/data/snapshots
        bind:
          create_host_path: true
  ka-garage-south:
    container_name: ka-garage-south
    environment:
      AGENT_AUTO_CLEAN: "true"
      AGENT_AUTO_CLEAN_MAX_SIZE: "102400"
      AGENT_CAPTURE_CONTINUOUS: "false"
      AGENT_CAPTURE_FRAGMENTED: "false"
      AGENT_CAPTURE_FRAGMENTED_DURATION: "8"
      AGENT_CAPTURE_IPCAMERA_RTSP: rtsp://xxx:xxx@192.168.1.62/defaultPrimary?streamType=u
      AGENT_CAPTURE_IPCAMERA_SUB_RTSP: rtsp://xxx:xxx@192.168.1.62/defaultPrimary?streamType=u
      AGENT_CAPTURE_MAXLENGTH: "600"
      AGENT_CAPTURE_PIXEL_CHANGE: "150"
      AGENT_CAPTURE_POSTRECORDING: "40"
      AGENT_CAPTURE_PRERECORDING: "20"
      AGENT_KEY: ""
      AGENT_MQTT_PASSWORD: ""
      AGENT_MQTT_URI: tcp://195.168.1.95:1883
      AGENT_MQTT_USERNAME: ""
      AGENT_NAME: garage-south
      AGENT_OFFLINE: "false"
      AGENT_PASSWORD: root
      AGENT_TIMEZONE: America/Edmonton
      AGENT_USERNAME: admin
    hostname: ka-garage-south
    image: kerberos/agent:latest
    networks:
      default: null
    ports:
      - mode: ingress
        target: 80
        published: "8082"
        protocol: tcp
    restart: unless-stopped
    volumes:
      - type: bind
        source: /Users/xxx/Desktop/kerberos/garage-south/recordings
        target: /home/agent/data/recordings
        bind:
          create_host_path: true
      - type: bind
        source: /Users/xxx/Desktop/kerberos/garage-south/snapshots
        target: /home/agent/data/snapshots
        bind:
          create_host_path: true
networks:
  default:
    name: kerberos-agent_default
instantdreams commented 1 year ago

Adding the /snapshots mount, removing the user entry, and grabbing a base configuration file enabled the container to start.

Before I document the process, I have a question for you, @cedricve - you said in your previous reply I could remove the config binding as I override it in the environment variables. I see in the example you included that ka-garage-north includes the user entry and ka-garage-south does not, but I cannot see anything else that has changed.

cedricve commented 1 year ago

I just updated the example (got rid of the user param). Forgot to remove it.

instantdreams commented 1 year ago

I started up five cameras and the Raspberry Pi I am testing this on got a bit slow. I restarted with just one camera and the container is running.

No environment variables were passed over. The agent is running as a default instance.

Here's the sanitised docker compose configuration:

$ docker compose --file /srv/kerberos-agent/docker-compose.yml config
name: kerberos-agent
services:
  ka-dreaming-front:
    container_name: ka-dreaming-front
    environment:
      AGENT_AUTO_CLEAN: "true"
      AGENT_AUTO_CLEAN_MAX_SIZE: "102400"
      AGENT_CAPTURE_CONTINUOUS: "false"
      AGENT_CAPTURE_FRAGMENTED: "false"
      AGENT_CAPTURE_FRAGMENTED_DURATION: "8"
      AGENT_CAPTURE_IPCAMERA_RTSP: rtsp://[details]/dreaming-front
      AGENT_CAPTURE_IPCAMERA_SUB_RTSP: rtsp://[details]/dreaming-front
      AGENT_CAPTURE_MAXLENGTH: "600"
      AGENT_CAPTURE_PIXEL_CHANGE: "150"
      AGENT_CAPTURE_POSTRECORDING: "40"
      AGENT_CAPTURE_PRERECORDING: "20"
      AGENT_KEY: ""
      AGENT_MQTT_PASSWORD: ""
      AGENT_MQTT_URI: tcp://195.168.1.95:1883
      AGENT_MQTT_USERNAME: ""
      AGENT_NAME: dreaming-front
      AGENT_OFFLINE: "false"
      AGENT_PASSWORD: [password]
      AGENT_TIMEZONE: America/Edmonton
      AGENT_USERNAME: admin
    hostname: id-edge1
    image: kerberos/agent:latest
    networks:
      default: null
    ports:
    - mode: ingress
      target: 80
      published: "8081"
      protocol: tcp
    restart: unless-stopped
    volumes:
    - type: bind
      source: /srv/kerberos-agent/dreaming-front/config
      target: /home/agent/data/config
      bind:
        create_host_path: true
    - type: bind
      source: /media/nvr/kerberos/dreaming-front/recordings
      target: /home/agent/data/recordings
      bind:
        create_host_path: true
    - type: bind
      source: /media/nvr/kerberos/dreaming-front/snapshots
      target: /home/agent/data/snapshots
      bind:
        create_host_path: true
networks:
  default:
    name: kerberos-agent_default

Here's the content of the container configuration file:

$ docker exec -it ka-dreaming-front sh
~ $ cat data/config/config.json
{
        "type": "",
        "key": "",
        "name": "agent",
        "time": "false",
        "offline": "false",
        "auto_clean": "true",
        "max_directory_size": 100,
        "timezone": "Africa/Ceuta",
        "capture": {
                "name": "",
                "ipcamera": {
                        "rtsp": "",
                        "sub_rtsp": "",
                        "fps": ""
                },
                "usbcamera": {
                        "device": ""
                },
                "raspicamera": {
                        "device": ""
                },
                "continuous": "false",
                "postrecording": 20,
                "prerecording": 10,
                "maxlengthrecording": 30,
                "transcodingwebrtc": "",
                "transcodingresolution": 0,
                "forwardwebrtc": "",
                "fragmented": "false",
                "fragmentedduration": 8,
                "pixelChangeThreshold": 150
        },
        "timetable": [
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                }
        ],
        "region": {
                "name": "",
                "rectangle": {
                        "x1": 0,
                        "y1": 0,
                        "x2": 800,
                        "y2": 640
                },
                "polygon": []
        },
        "cloud": "s3",
        "s3": {
                "proxyuri": "http://proxy.kerberos.io",
                "bucket": "kerberosaccept",
                "region": "eu-west1"
        },
        "kstorage": {},
        "mqtturi": "tcp://mqtt.kerberos.io:1883",
        "mqtt_username": "",
        "mqtt_password": "",
        "stunuri": "stun:turn.kerberos.io:8443",
        "turnuri": "turn:turn.kerberos.io:8443",
        "turn_username": "username1",
        "turn_password": "password1",
        "heartbeaturi": "",
        "hub_uri": "https://api.cloud.kerberos.io",
        "hub_key": "",
        "hub_private_key": "",
        "hub_site": "",
        "condition_uri": ""

I was able to log in using the username and password I set in the environment variables, but no other environment variables have been propagated through to the configuration file.

Here are the container logs:

$ docker compose --file /srv/kerberos-agent/docker-compose.yml logs
ka-dreaming-front  | {"level":"info","msg":"Successfully Opened config.json from opensource","time":"2022-12-20T19:44:08+01:00"}
ka-dreaming-front  | {"level":"info","msg":"Main: something went wrong while trying to store key: [key]","time":"2022-12-20T11:44:08-07:00"}
ka-dreaming-front  | [GIN-debug] [WARNING] Creating an Engine instance with the Logger and Recovery middleware already attached.
ka-dreaming-front  |
ka-dreaming-front  | [GIN-debug] [WARNING] Running in "debug" mode. Switch to "release" mode in production.
ka-dreaming-front  |  - using env:      export GIN_MODE=release
ka-dreaming-front  |  - using code:     gin.SetMode(gin.ReleaseMode)
ka-dreaming-front  |
ka-dreaming-front  | [GIN-debug] GET    /debug/pprof/             --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers)
ka-dreaming-front  | [GIN-debug] GET    /debug/pprof/cmdline      --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers)
ka-dreaming-front  | [GIN-debug] GET    /debug/pprof/profile      --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers)
ka-dreaming-front  | [GIN-debug] POST   /debug/pprof/symbol       --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers)
ka-dreaming-front  | [GIN-debug] GET    /debug/pprof/symbol       --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers)
ka-dreaming-front  | [GIN-debug] GET    /debug/pprof/trace        --> github.com/gin-gonic/gin.WrapF.func1 (3 handlers)
ka-dreaming-front  | [GIN-debug] GET    /debug/pprof/allocs       --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers)
ka-dreaming-front  | [GIN-debug] GET    /debug/pprof/block        --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers)
ka-dreaming-front  | [GIN-debug] GET    /debug/pprof/goroutine    --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers)
ka-dreaming-front  | [GIN-debug] GET    /debug/pprof/heap         --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers)
ka-dreaming-front  | [GIN-debug] GET    /debug/pprof/mutex        --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers)
ka-dreaming-front  | [GIN-debug] GET    /debug/pprof/threadcreate --> github.com/gin-gonic/gin.WrapH.func1 (3 handlers)
ka-dreaming-front  | {"level":"info","msg":"RunAgent: opening RTSP stream","time":"2022-12-20T11:44:08-07:00"}
ka-dreaming-front  | [GIN-debug] GET    /swagger/*any             --> github.com/swaggo/gin-swagger.CustomWrapHandler.func1 (4 handlers)
ka-dreaming-front  | [GIN-debug] GET    /config                   --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func1 (4 handlers)
ka-dreaming-front  | [GIN-debug] POST   /config                   --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func2 (4 handlers)
ka-dreaming-front  | [GIN-debug] POST   /api/login                --> github.com/appleboy/gin-jwt/v2.(*GinJWTMiddleware).LoginHandler-fm (4 handlers)
ka-dreaming-front  | [GIN-debug] GET    /api/dashboard            --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func3 (4 handlers)
ka-dreaming-front  | [GIN-debug] POST   /api/latest-events        --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func4 (4 handlers)
ka-dreaming-front  | [GIN-debug] GET    /api/days                 --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func5 (4 handlers)
ka-dreaming-front  | [GIN-debug] GET    /api/stream               --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func6 (4 handlers)
ka-dreaming-front  | [GIN-debug] GET    /api/config               --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func7 (4 handlers)
ka-dreaming-front  | [GIN-debug] POST   /api/config               --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func8 (4 handlers)
ka-dreaming-front  | [GIN-debug] GET    /api/restart              --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func9 (4 handlers)
ka-dreaming-front  | [GIN-debug] GET    /api/stop                 --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func10 (4 handlers)
ka-dreaming-front  | [GIN-debug] POST   /api/camera/verify/:streamType --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func11 (4 handlers)
ka-dreaming-front  | [GIN-debug] POST   /api/hub/verify           --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func12 (4 handlers)
ka-dreaming-front  | [GIN-debug] POST   /api/persistence/verify   --> github.com/kerberos-io/agent/machinery/src/routers/http.AddRoutes.func13 (4 handlers)
ka-dreaming-front  | [GIN-debug] GET    /file/*filepath           --> github.com/gin-gonic/gin.(*RouterGroup).createStaticHandler.func1 (9 handlers)
ka-dreaming-front  | [GIN-debug] HEAD   /file/*filepath           --> github.com/gin-gonic/gin.(*RouterGroup).createStaticHandler.func1 (9 handlers)
ka-dreaming-front  | [GIN-debug] [WARNING] You trusted all proxies, this is NOT safe. We recommend you to set a value.
ka-dreaming-front  | Please check https://pkg.go.dev/github.com/gin-gonic/gin#readme-don-t-trust-all-proxies for details.
ka-dreaming-front  | [GIN-debug] Listening and serving HTTP on :80
ka-dreaming-front  | {"level":"info","msg":"ControlAgent: Number of packets read 0","time":"2022-12-20T11:44:08-07:00"}
ka-dreaming-front  | {"level":"error","msg":"Something went wrong while opening RTSP: EOF","time":"2022-12-20T11:44:08-07:00"}

If left up, the error "Something went wrong while opening RTSP: open : no such file or directory" repeats, as there is no rtsp entry in the configuration file.

cedricve commented 1 year ago

As mentioned in previous comment, you should also get rid of the config binding. What happens in the system is that your RTSP url in the environment variable AGENT_CAPTURE_IPCAMERA_RTSP overrides the configuration file (even if it is blank). Not sure why you cannot connect to your camera stream. Please verify if its valid.

cedricve commented 1 year ago

Also make sure you are pulling the latest as environment variables where only added last week.

instantdreams commented 1 year ago

I removed the config binding. This means there is no need to get a blank config.json file as nothing is bound. I am using the latest version. We have a running container with an stream active.

However, any changes to the configuration are NOT persisted.

Here's the configuration file before changes:

$ docker exec -it ka-garage-south sh
~ $ cat data/config/config.json
{
        "type": "",
        "key": "[key]",
        "name": "garage-south",
        "time": "false",
        "offline": "false",
        "auto_clean": "true",
        "max_directory_size": 102400,
        "timezone": "America/Edmonton",
        "capture": {
                "name": "",
                "ipcamera": {
                        "rtsp": "rtsp://[details]/cam/realmonitor?channel=1\u0026subtype=0",
                        "sub_rtsp": "rtsp://[details]/cam/realmonitor?channel=1\u0026subtype=1",
                        "fps": ""
                },
                "usbcamera": {
                        "device": ""
                },
                "raspicamera": {
                        "device": ""
                },
                "continuous": "false",
                "postrecording": 40,
                "prerecording": 20,
                "maxlengthrecording": 600,
                "transcodingwebrtc": "",
                "transcodingresolution": 0,
                "forwardwebrtc": "",
                "fragmented": "false",
                "fragmentedduration": 8,
                "pixelChangeThreshold": 150
        },
        "timetable": [
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                }
        ],
        "region": {
                "name": "",
                "rectangle": {
                        "x1": 0,
                        "y1": 0,
                        "x2": 800,
                        "y2": 640
                },
                "polygon": []
        },
        "cloud": "s3",
        "s3": {
                "proxyuri": "http://proxy.kerberos.io",
                "bucket": "kerberosaccept",
                "region": "eu-west1"
        },
        "kstorage": {},
        "mqtturi": "tcp://195.168.1.95:1883",
        "mqtt_username": "",
        "mqtt_password": "",
        "stunuri": "stun:turn.kerberos.io:8443",
        "turnuri": "turn:turn.kerberos.io:8443",
        "turn_username": "username1",
        "turn_password": "password1",
        "heartbeaturi": "",
        "hub_uri": "https://api.cloud.kerberos.io",
        "hub_key": "",
        "hub_private_key": "",
        "hub_site": "",
        "condition_uri": ""

Here's the configuration file after changes:

$ docker exec -it ka-garage-south sh
~ $ cat data/config/config.json
{
        "type": "",
        "key": "[key]",
        "name": "garage-south",
        "time": "false",
        "offline": "false",
        "auto_clean": "true",
        "max_directory_size": 102400,
        "timezone": "America/Edmonton",
        "capture": {
                "name": "",
                "ipcamera": {
                        "rtsp": "rtsp://[details]/cam/realmonitor?channel=1\u0026subtype=0",
                        "sub_rtsp": "rtsp://[details]/cam/realmonitor?channel=1\u0026subtype=1",
                        "fps": ""
                },
                "usbcamera": {
                        "device": ""
                },
                "raspicamera": {
                        "device": ""
                },
                "continuous": "false",
                "postrecording": 40,
                "prerecording": 20,
                "maxlengthrecording": 600,
                "transcodingwebrtc": "",
                "transcodingresolution": 0,
                "forwardwebrtc": "",
                "fragmented": "false",
                "fragmentedduration": 8,
                "pixelChangeThreshold": 150
        },
        "timetable": [
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                },
                {
                        "start1": 0,
                        "end1": 43199,
                        "start2": 43200,
                        "end2": 86400
                }
        ],
        "region": {
                "name": "",
                "rectangle": {
                        "x1": 0,
                        "y1": 0,
                        "x2": 800,
                        "y2": 640
                },
                "polygon": []
        },
        "cloud": "s3",
        "s3": {
                "bucket": "kerberosaccept",
                "region": "eu-west1"
        },
        "kstorage": {},
        "mqtturi": "tcp://195.168.1.95:1883",
        "mqtt_username": "",
        "mqtt_password": "",
        "stunuri": "stun:turn.kerberos.io:8443",
        "turnuri": "turn:turn.kerberos.io:8443",
        "turn_username": "username1",
        "turn_password": "password1",
        "heartbeaturi": "",
        "hub_uri": "",
        "hub_key": "",
        "hub_private_key": "",
        "hub_site": "",
        "condition_uri": ""

There is no difference. I changed items that were not passed as environment variables - specifically Conditions | Regions of interest | polygon - and nothing was altered.

It may be that this is expected when environment variables override the configuration file. In this case, it may be better for advanced users to mount and set up configuration files for each agent.

Note that timezone is specified in the configuration file but does not appear on the configuration GUI. The GUI appears to represent a subset of the tz database so I'll have to adjust any settings to match that.

Also, thanks @cedricve for working with me on this.

cedricve commented 1 year ago

I think I've misunderstood you ;) No this should not be the case, you should still be able to modify the configuration. However environment variables will always override (if available), so for your use case, where region of interest is not set, you should still be able to define it through the UI. Let me have a look as well.

cedricve commented 1 year ago

I just tested with updating the region of interest, and I noticed it might take a few seconds before it's visible. But the config file under the hood was modified.

{
  "id": "1671564777871",
  "coordinates": [
    {
      "x": 68.66379310344828,
      "y": 62.46077586206897
    },
    {
      "x": 544.2362068965517,
      "y": 62.46077586206897
    },
    {
      "x": 544.2362068965517,
      "y": 341.16012931034487
    },
    {
      "x": 68.66379310344828,
      "y": 341.16012931034487
    }
  ]
}
instantdreams commented 1 year ago

Configuration updated to set boundary conditions. GUI confirmed update took place. Log file viewed:

ka-garage-south  | [GIN] 2022/12/20 - 19:38:25 | 200 |     369.484µs |   192.168.1.106 | GET      "/api/dashboard"
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:26-07:00"}
ka-garage-south  | {"level":"error","msg":"HandleHeartBeat: (400) Something went wrong while sending to Kerberos Hub.","time":"2022-12-20T12:38:26-07:00"}
ka-garage-south  | {"level":"info","msg":"HandleStream: packet size 623004","time":"2022-12-20T12:38:26-07:00"}
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:28-07:00"}
ka-garage-south  | {"level":"info","msg":"ControlAgent: Number of packets read 754","time":"2022-12-20T12:38:28-07:00"}
ka-garage-south  | {"level":"info","msg":"HandleStream: packet size 619080","time":"2022-12-20T12:38:28-07:00"}
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:30-07:00"}
ka-garage-south  | {"level":"info","msg":"HandleStream: packet size 623841","time":"2022-12-20T12:38:30-07:00"}
ka-garage-south  | [GIN] 2022/12/20 - 19:38:30 | 200 |     450.854µs |   192.168.1.106 | GET      "/api/dashboard"
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:32-07:00"}
ka-garage-south  | {"level":"info","msg":"HandleStream: packet size 622118","time":"2022-12-20T12:38:32-07:00"}
ka-garage-south  | {"level":"info","msg":"ControlAgent: Number of packets read 757","time":"2022-12-20T12:38:33-07:00"}
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:33-07:00"}
ka-garage-south  | {"level":"info","msg":"HandleStream: packet size 611980","time":"2022-12-20T12:38:34-07:00"}
ka-garage-south  | [GIN] 2022/12/20 - 19:38:35 | 200 |     797.893µs |   192.168.1.106 | GET      "/api/dashboard"
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:36-07:00"}
ka-garage-south  | {"level":"info","msg":"HandleStream: packet size 618717","time":"2022-12-20T12:38:36-07:00"}
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:38-07:00"}
ka-garage-south  | {"level":"info","msg":"ControlAgent: Number of packets read 759","time":"2022-12-20T12:38:38-07:00"}
ka-garage-south  | {"level":"info","msg":"HandleStream: packet size 622289","time":"2022-12-20T12:38:38-07:00"}
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:39-07:00"}
ka-garage-south  | {"level":"info","msg":"HandleStream: packet size 622486","time":"2022-12-20T12:38:40-07:00"}
ka-garage-south  | [GIN] 2022/12/20 - 19:38:40 | 200 |    1.666713ms |   192.168.1.106 | GET      "/api/dashboard"
ka-garage-south  | {"level":"error","msg":"HandleHeartBeat: (400) Something went wrong while sending to Kerberos Hub.","time":"2022-12-20T12:38:41-07:00"}
ka-garage-south  | [GIN] 2022/12/20 - 19:38:41 | 200 |     285.761µs |       127.0.0.1 | GET      "/"
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:42-07:00"}
ka-garage-south  | {"level":"info","msg":"HandleStream: packet size 623292","time":"2022-12-20T12:38:42-07:00"}
ka-garage-south  | {"level":"info","msg":"ControlAgent: Number of packets read 762","time":"2022-12-20T12:38:43-07:00"}
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:44-07:00"}
ka-garage-south  | {"level":"info","msg":"HandleStream: packet size 624047","time":"2022-12-20T12:38:44-07:00"}
ka-garage-south  | [GIN] 2022/12/20 - 19:38:45 | 200 |     248.131µs |   192.168.1.106 | GET      "/api/dashboard"
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:46-07:00"}
ka-garage-south  | {"level":"info","msg":"HandleStream: packet size 620309","time":"2022-12-20T12:38:46-07:00"}
ka-garage-south  | [GIN] 2022/12/20 - 19:38:46 | 200 |    2.014622ms |   192.168.1.106 | POST     "/api/config"
ka-garage-south  | {"level":"info","msg":"WriteToTrack: stop writing to track.","time":"2022-12-20T12:38:47-07:00"}
ka-garage-south  | {"level":"info","msg":"FindMotion: Number of changes detected:0","time":"2022-12-20T12:38:47-07:00"}
ka-garage-south  | {"level":"info","msg":"RunAgent: waiting 3 seconds to make sure everything is properly closed.","time":"2022-12-20T12:38:47-07:00"}
ka-garage-south  | {"level":"info","msg":"ControlAgent: Number of packets read 764","time":"2022-12-20T12:38:48-07:00"}

No evidence of any changes there.

Configuration changed Settings | Recordings | Continuous recording: Enabled. Saved, GUI confirmed "Your configuration have been updated successfully.". Log file viewed:

ka-garage-south  | {"level":"info","msg":"WriteToTrack: stop writing to track.","time":"2022-12-20T12:39:53-07:00"}
ka-garage-south  | unexpected fault address 0x7f5d2eb000
ka-garage-south  | fatal error: fault
ka-garage-south  | [signal SIGSEGV: segmentation violation code=0x2 addr=0x7f5d2eb000 pc=0x5d6874]
ka-garage-south  |
ka-garage-south  | goroutine 661 [running]:
ka-garage-south  | runtime.throw({0xef1d0a?, 0x0?})
ka-garage-south  |      /usr/local/go/src/runtime/panic.go:992 +0x50 fp=0x4000e63530 sp=0x4000e63500 pc=0x4521a0
ka-garage-south  | runtime.sigpanic()
ka-garage-south  |      /usr/local/go/src/runtime/signal_unix.go:825 +0x1a4 fp=0x4000e63560 sp=0x4000e63530 pc=0x4693c4
ka-garage-south  | image.(*YCbCr).YCbCrAt(0x4000e63688, 0xda0, 0x5fc)
ka-garage-south  |      /usr/local/go/src/image/ycbcr.go:86 +0xa4 fp=0x4000e635b0 sp=0x4000e63570 pc=0x5d6874
ka-garage-south  | image.(*YCbCr).At(...)
ka-garage-south  |      /usr/local/go/src/image/ycbcr.go:71
ka-garage-south  | github.com/kerberos-io/agent/machinery/src/computervision.ToRGB8({{0x7f5cd4de60, 0x7e9000, 0x7e9000}, {0x7f5c2388c0, 0x1fa400, 0x1fa400}, {0x7f5c437070, 0x1fa400, 0x1fa400}, 0xf00, ...})
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/src/computervision/main.go:90 +0x170 fp=0x4000e63680 sp=0x4000e635b0 pc=0xa79960
ka-garage-south  | github.com/kerberos-io/agent/machinery/src/computervision.GetImage({0x1, 0x0, 0x0, 0xe7d327180, {0x4005d00000, 0x991a6, 0x991a6}}, 0x96?, 0x400036e5e8?)
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/src/computervision/main.go:67 +0x1c8 fp=0x4000e63920 sp=0x4000e63680 pc=0xa79668
ka-garage-south  | github.com/kerberos-io/agent/machinery/src/computervision.ProcessMotion(0x40002bc6e0?, 0x40001da000, 0x4000482200, {0x108e8b8, 0x400042d200}, 0x40031d4000?, 0x400036e5e8?)
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/src/computervision/main.go:200 +0x580 fp=0x4000e63f80 sp=0x4000e63920 pc=0xa79fe0
ka-garage-south  | github.com/kerberos-io/agent/machinery/src/components.RunAgent.func8()
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/src/components/Kerberos.go:157 +0x44 fp=0x4000e63fd0 sp=0x4000e63f80 pc=0xbd9324
ka-garage-south  | runtime.goexit()
ka-garage-south  |      /usr/local/go/src/runtime/asm_arm64.s:1259 +0x4 fp=0x4000e63fd0 sp=0x4000e63fd0 pc=0x484bc4
ka-garage-south  | created by github.com/kerberos-io/agent/machinery/src/components.RunAgent
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/src/components/Kerberos.go:157 +0x914
ka-garage-south  |
ka-garage-south  | goroutine 1 [IO wait]:
ka-garage-south  | internal/poll.runtime_pollWait(0x7f7d462ed8, 0x72)
ka-garage-south  |      /usr/local/go/src/runtime/netpoll.go:302 +0xa4
ka-garage-south  | internal/poll.(*pollDesc).wait(0x4000482500?, 0x4cc254?, 0x0)
ka-garage-south  |      /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x2c
ka-garage-south  | internal/poll.(*pollDesc).waitRead(...)
ka-garage-south  |      /usr/local/go/src/internal/poll/fd_poll_runtime.go:88
ka-garage-south  | internal/poll.(*FD).Accept(0x4000482500)
ka-garage-south  |      /usr/local/go/src/internal/poll/fd_unix.go:614 +0x1d8
ka-garage-south  | net.(*netFD).accept(0x4000482500)
ka-garage-south  |      /usr/local/go/src/net/fd_unix.go:172 +0x28
ka-garage-south  | net.(*TCPListener).accept(0x400000eb40)
ka-garage-south  |      /usr/local/go/src/net/tcpsock_posix.go:139 +0x2c
ka-garage-south  | net.(*TCPListener).Accept(0x400000eb40)
ka-garage-south  |      /usr/local/go/src/net/tcpsock.go:288 +0x30
ka-garage-south  | net/http.(*Server).Serve(0x400056a0e0, {0x1089820, 0x400000eb40})
ka-garage-south  |      /usr/local/go/src/net/http/server.go:3039 +0x300
ka-garage-south  | net/http.(*Server).ListenAndServe(0x400056a0e0)
ka-garage-south  |      /usr/local/go/src/net/http/server.go:2968 +0x88
ka-garage-south  | net/http.ListenAndServe(...)
ka-garage-south  |      /usr/local/go/src/net/http/server.go:3222
ka-garage-south  | github.com/gin-gonic/gin.(*Engine).Run(0x400043eb60, {0x4000131a90, 0x1, 0x1})
ka-garage-south  |      /go/pkg/mod/github.com/gin-gonic/gin@v1.8.1/gin.go:382 +0x1d4
ka-garage-south  | github.com/kerberos-io/agent/machinery/src/routers/http.StartServer(0x40001da000, 0x1?)
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/src/routers/http/Server.go:69 +0x600
ka-garage-south  | github.com/kerberos-io/agent/machinery/src/routers.StartWebserver(...)
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/src/routers/main.go:9
ka-garage-south  | main.main()
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/main.go:91 +0x378
ka-garage-south  |
ka-garage-south  | goroutine 7 [syscall]:
ka-garage-south  | github.com/kerberos-io/joy4/cgo/ffmpeg._Cfunc_avcodec_close(0x7f5c000b60)
ka-garage-south  |      _cgo_gotypes.go:1039 +0x3c
ka-garage-south  | github.com/kerberos-io/joy4/cgo/ffmpeg.freeFFCtx.func2(0x40031cfaa0?)
ka-garage-south  |      /go/pkg/mod/github.com/kerberos-io/joy4@v1.0.33/cgo/ffmpeg/ffmpeg.go:66 +0x44
ka-garage-south  | github.com/kerberos-io/joy4/cgo/ffmpeg.freeFFCtx(0x40031d4030)
ka-garage-south  |      /go/pkg/mod/github.com/kerberos-io/joy4@v1.0.33/cgo/ffmpeg/ffmpeg.go:66 +0x3c
ka-garage-south  | github.com/kerberos-io/joy4/cgo/ffmpeg.(*VideoDecoder).Close(...)
ka-garage-south  |      /go/pkg/mod/github.com/kerberos-io/joy4@v1.0.33/cgo/ffmpeg/video.go:844
ka-garage-south  | github.com/kerberos-io/agent/machinery/src/components.RunAgent(0x40001da000, 0x4000482200, {0x107d5f8?, 0x0?, 0x1f36540?})
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/src/components/Kerberos.go:208 +0xfb4
ka-garage-south  | github.com/kerberos-io/agent/machinery/src/components.Bootstrap(0x0?, 0x4000482200)
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/src/components/Kerberos.go:65 +0x398
ka-garage-south  | created by main.main
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/main.go:88 +0x36c
ka-garage-south  |
ka-garage-south  | goroutine 50 [sleep]:
ka-garage-south  | time.Sleep(0x12a05f200)
ka-garage-south  |      /usr/local/go/src/runtime/time.go:194 +0x124
ka-garage-south  | github.com/kerberos-io/agent/machinery/src/components.ControlAgent.func1()
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/src/components/Kerberos.go:256 +0x54
ka-garage-south  | created by github.com/kerberos-io/agent/machinery/src/components.ControlAgent
ka-garage-south  |      /go/src/github.com/kerberos-io/agent/machinery/src/components/Kerberos.go:228 +0x9c
ka-garage-south  |
ka-garage-south  | goroutine 569 [IO wait]:
ka-garage-south  | internal/poll.runtime_pollWait(0x7f7d462c08, 0x72)
ka-garage-south  |      /usr/local/go/src/runtime/netpoll.go:302 +0xa4
ka-garage-south  | internal/poll.(*pollDesc).wait(0x4000162180?, 0x40018a3000?, 0x0)
ka-garage-south  |      /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x2c
ka-garage-south  | internal/poll.(*pollDesc).waitRead(...)
ka-garage-south  |      /usr/local/go/src/internal/poll/fd_poll_runtime.go:88
ka-garage-south  | internal/poll.(*FD).Read(0x4000162180, {0x40018a3000, 0x1000, 0x1000})
ka-garage-south  |      /usr/local/go/src/internal/poll/fd_unix.go:167 +0x1ec
ka-garage-south  | net.(*netFD).Read(0x4000162180, {0x40018a3000?, 0x4000e2b8e8?, 0x757460?})
ka-garage-south  |      /usr/local/go/src/net/fd_posix.go:55 +0x2c
ka-garage-south  | net.(*conn).Read(0x4000010008, {0x40018a3000?, 0x4000e2b928?, 0x72?})
ka-garage-south  |      /usr/local/go/src/net/net.go:183 +0x38
ka-garage-south  | net/http.(*connReader).Read(0x40000b8240, {0x40018a3000, 0x1000, 0x1000})
ka-garage-south  |      /usr/local/go/src/net/http/server.go:780 +0x250
ka-garage-south  | bufio.(*Reader).fill(0x400009ab40)
ka-garage-south  |      /usr/local/go/src/bufio/bufio.go:106 +0x100
ka-garage-south  | bufio.(*Reader).Peek(0x400009ab40, 0x4)
ka-garage-south  |      /usr/local/go/src/bufio/bufio.go:144 +0x6c
ka-garage-south  | net/http.(*conn).readRequest(0x40003e2000, {0x108a4c8, 0x4000368040})
ka-garage-south  |      /usr/local/go/src/net/http/server.go:985 +0x1ac
ka-garage-south  | net/http.(*conn).serve(0x40003e2000, {0x108a570, 0x40000b9620})
ka-garage-south  |      /usr/local/go/src/net/http/server.go:1891 +0x2e4
ka-garage-south  | created by net/http.(*Server).Serve
ka-garage-south  |      /usr/local/go/src/net/http/server.go:3071 +0x448
ka-garage-south  |
ka-garage-south  | goroutine 634 [sleep]:
ka-garage-south  | time.Sleep(0x6fc23ac00)
ka-garage-south  |      /usr/local/go/src/runtime/time.go:194 +0x124
ka-garage-south  | github.com/eclipse/paho%2emqtt%2egolang.(*client).Connect.func1()
ka-garage-south  |      /go/pkg/mod/github.com/eclipse/paho.mqtt.golang@v1.4.1/client.go:286 +0x248
ka-garage-south  | created by github.com/eclipse/paho%2emqtt%2egolang.(*client).Connect
ka-garage-south  |      /go/pkg/mod/github.com/eclipse/paho.mqtt.golang@v1.4.1/client.go:272 +0x18c
ka-garage-south  | {"level":"info","msg":"Successfully Opened config.json from opensource","time":"2022-12-20T20:39:56+01:00"}
ka-garage-south  | {"level":"info","msg":"Main: updated unique key for agent to: [key]","time":"2022-12-20T12:39:56-07:00"}

Settings not updated.

Attempted this a second time. Configuration file WAS updated, no errors in log file.

I hate intermittent errors.

Here's my current configuration for reference.

docker-compose.yml

version: "3.9"
services:
  ka-garage-south:
    image: "kerberos/agent:latest"
    container_name: ka-garage-south
    hostname: ${HOSTNAME}
    ports:
      - "8082:80"
    env_file:
      - agent-settings.env
    environment:
      - AGENT_NAME=garage-south
      - AGENT_CAPTURE_IPCAMERA_RTSP=${IPCAMERA_RTSP_GARAGE_SOUTH}
      - AGENT_CAPTURE_IPCAMERA_SUB_RTSP=${IPCAMERA_SUB_RTSP_GARAGE_SOUTH}
    volumes:
      - ${RECORDINGS}/garage-south/recordings:/home/agent/data/recordings:rw
      - ${RECORDINGS}/garage-south/snapshots:/home/agent/data/snapshots:rw
    restart: unless-stopped

agent-settings.env

AGENT_USERNAME="admin"
AGENT_PASSWORD=${PASSWORD}
AGENT_KEY=""
#AGENT_NAME="${AGENT_NAME}"
AGENT_TIMEZONE="America/Edmonton"
AGENT_OFFLINE="false"
AGENT_AUTO_CLEAN="true"
AGENT_AUTO_CLEAN_MAX_SIZE="102400"
#AGENT_CAPTURE_IPCAMERA_RTSP="${AGENT_CAPTURE_IPCAMERA_RTSP}"
#AGENT_CAPTURE_IPCAMERA_SUB_RTSP="${AGENT_CAPTURE_IPCAMERA_SUB_RTSP}"
AGENT_CAPTURE_CONTINUOUS="false"
AGENT_CAPTURE_PRERECORDING="20"
AGENT_CAPTURE_POSTRECORDING="40"
AGENT_CAPTURE_MAXLENGTH="600"
AGENT_CAPTURE_PIXEL_CHANGE="150"
AGENT_CAPTURE_FRAGMENTED="false"
AGENT_CAPTURE_FRAGMENTED_DURATION="8"
#AGENT_CLOUD="s3"
#AGENT_HUB_URI="https://api.cloud.kerberos.io"
#AGENT_HUB_KEY=""
#AGENT_HUB_PRIVATE_KEY=""
#AGENT_HUB_USERNAME=""
#AGENT_HUB_SITE=""
AGENT_MQTT_URI="tcp://195.168.1.95:1883"
AGENT_MQTT_USERNAME=""
AGENT_MQTT_PASSWORD=""
#AGENT_STUN_URI="stun:turn.kerberos.io:8443"
#AGENT_TURN_URI="turn:turn.kerberos.io:8443"
#AGENT_TURN_USERNAME="username1"
#AGENT_TURN_PASSWORD="password1"
#AGENT_KERBEROSVAULT_URI=""
#AGENT_KERBEROSVAULT_ACCESS_KEY=""
#AGENT_KERBEROSVAULT_SECRET_KEY=""
#AGENT_KERBEROSVAULT_PROVIDER=""
#AGENT_KERBEROSVAULT_DIRECTORY=""

.env

# Timezone from https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
TZ=America/Edmonton
# Directory locations
RECORDINGS=/media/nvr/kerberos
# Container specifics
HOSTNAME=id-edge1
PASSWORD=[password]
IPCAMERA_RTSP_GARAGE_SOUTH=rtsp://[user]:[pass]@192.168.1.83:554/cam/realmonitor?channel=1&subtype=0
IPCAMERA_SUB_RTSP_GARAGE_SOUTH=rtsp://[user]:[pass]@192.168.1.83:554/cam/realmonitor?channel=1&subtype=1

Thanks for looking into this.

instantdreams commented 1 year ago

For whatever reason it's now saving changes to everything other than the polygon settings in Conditions.

cedricve commented 1 year ago

ka-garage-south | {"level":"info","msg":"WriteToTrack: stop writing to track.","time":"2022-12-20T12:39:53-07:00"} ka-garage-south | unexpected fault address 0x7f5d2eb000 ka-garage-south | fatal error: fault ka-garage-south | [signal SIGSEGV: segmentation violation code=0x2 addr=0x7f5d2eb000 pc=0x5d6874] ka-garage-south | ka-garage-south | goroutine 661 [running]: ka-garage-south | runtime.throw({0xef1d0a?, 0x0?}) ka-garage-south | /usr/local/go/src/runtime/panic.go:992 +0x50 fp=0x4000e63530 sp=0x4000e63500 pc=0x4521a0 ka-garage-south | runtime.sigpanic() ka-garage-south | /usr/local/go/src/runtime/signal_unix.go:825 +0x1a4 fp=0x4000e63560 sp=0x4000e63530 pc=0x4693c4 ka-garage-south | image.(YCbCr).YCbCrAt(0x4000e63688, 0xda0, 0x5fc) ka-garage-south | /usr/local/go/src/image/ycbcr.go:86 +0xa4 fp=0x4000e635b0 sp=0x4000e63570 pc=0x5d6874 ka-garage-south | image.(YCbCr).At(...) ka-garage-south | /usr/local/go/src/image/ycbcr.go:71 ka-garage-south | github.com/kerberos-io/agent/machinery/src/computervision.ToRGB8({{0x7f5cd4de60, 0x7e9000, 0x7e9000}, {0x7f5c2388c0, 0x1fa400, 0x1fa400}, {0x7f5c437070, 0x1fa400, 0x1fa400}, 0xf00, ...}) ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/src/computervision/main.go:90 +0x170 fp=0x4000e63680 sp=0x4000e635b0 pc=0xa79960 ka-garage-south | github.com/kerberos-io/agent/machinery/src/computervision.GetImage({0x1, 0x0, 0x0, 0xe7d327180, {0x4005d00000, 0x991a6, 0x991a6}}, 0x96?, 0x400036e5e8?) ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/src/computervision/main.go:67 +0x1c8 fp=0x4000e63920 sp=0x4000e63680 pc=0xa79668 ka-garage-south | github.com/kerberos-io/agent/machinery/src/computervision.ProcessMotion(0x40002bc6e0?, 0x40001da000, 0x4000482200, {0x108e8b8, 0x400042d200}, 0x40031d4000?, 0x400036e5e8?) ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/src/computervision/main.go:200 +0x580 fp=0x4000e63f80 sp=0x4000e63920 pc=0xa79fe0 ka-garage-south | github.com/kerberos-io/agent/machinery/src/components.RunAgent.func8() ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/src/components/Kerberos.go:157 +0x44 fp=0x4000e63fd0 sp=0x4000e63f80 pc=0xbd9324 ka-garage-south | runtime.goexit() ka-garage-south | /usr/local/go/src/runtime/asm_arm64.s:1259 +0x4 fp=0x4000e63fd0 sp=0x4000e63fd0 pc=0x484bc4 ka-garage-south | created by github.com/kerberos-io/agent/machinery/src/components.RunAgent ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/src/components/Kerberos.go:157 +0x914 ka-garage-south | ka-garage-south | goroutine 1 [IO wait]: ka-garage-south | internal/poll.runtime_pollWait(0x7f7d462ed8, 0x72) ka-garage-south | /usr/local/go/src/runtime/netpoll.go:302 +0xa4 ka-garage-south | internal/poll.(pollDesc).wait(0x4000482500?, 0x4cc254?, 0x0) ka-garage-south | /usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x2c ka-garage-south | internal/poll.(pollDesc).waitRead(...) ka-garage-south | /usr/local/go/src/internal/poll/fd_poll_runtime.go:88 ka-garage-south | internal/poll.(FD).Accept(0x4000482500) ka-garage-south | /usr/local/go/src/internal/poll/fd_unix.go:614 +0x1d8 ka-garage-south | net.(netFD).accept(0x4000482500) ka-garage-south | /usr/local/go/src/net/fd_unix.go:172 +0x28 ka-garage-south | net.(TCPListener).accept(0x400000eb40) ka-garage-south | /usr/local/go/src/net/tcpsock_posix.go:139 +0x2c ka-garage-south | net.(TCPListener).Accept(0x400000eb40) ka-garage-south | /usr/local/go/src/net/tcpsock.go:288 +0x30 ka-garage-south | net/http.(Server).Serve(0x400056a0e0, {0x1089820, 0x400000eb40}) ka-garage-south | /usr/local/go/src/net/http/server.go:3039 +0x300 ka-garage-south | net/http.(Server).ListenAndServe(0x400056a0e0) ka-garage-south | /usr/local/go/src/net/http/server.go:2968 +0x88 ka-garage-south | net/http.ListenAndServe(...) ka-garage-south | /usr/local/go/src/net/http/server.go:3222 ka-garage-south | github.com/gin-gonic/gin.(Engine).Run(0x400043eb60, {0x4000131a90, 0x1, 0x1}) ka-garage-south | /go/pkg/mod/github.com/gin-gonic/gin@v1.8.1/gin.go:382 +0x1d4 ka-garage-south | github.com/kerberos-io/agent/machinery/src/routers/http.StartServer(0x40001da000, 0x1?) ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/src/routers/http/Server.go:69 +0x600 ka-garage-south | github.com/kerberos-io/agent/machinery/src/routers.StartWebserver(...) ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/src/routers/main.go:9 ka-garage-south | main.main() ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/main.go:91 +0x378 ka-garage-south | ka-garage-south | goroutine 7 [syscall]: ka-garage-south | github.com/kerberos-io/joy4/cgo/ffmpeg._Cfunc_avcodec_close(0x7f5c000b60) ka-garage-south | _cgo_gotypes.go:1039 +0x3c ka-garage-south | github.com/kerberos-io/joy4/cgo/ffmpeg.freeFFCtx.func2(0x40031cfaa0?) ka-garage-south | /go/pkg/mod/github.com/kerberos-io/joy4@v1.0.33/cgo/ffmpeg/ffmpeg.go:66 +0x44 ka-garage-south | github.com/kerberos-io/joy4/cgo/ffmpeg.freeFFCtx(0x40031d4030) ka-garage-south | /go/pkg/mod/github.com/kerberos-io/joy4@v1.0.33/cgo/ffmpeg/ffmpeg.go:66 +0x3c ka-garage-south | github.com/kerberos-io/joy4/cgo/ffmpeg.(VideoDecoder).Close(...) ka-garage-south | /go/pkg/mod/github.com/kerberos-io/joy4@v1.0.33/cgo/ffmpeg/video.go:844 ka-garage-south | github.com/kerberos-io/agent/machinery/src/components.RunAgent(0x40001da000, 0x4000482200, {0x107d5f8?, 0x0?, 0x1f36540?}) ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/src/components/Kerberos.go:208 +0xfb4 ka-garage-south | github.com/kerberos-io/agent/machinery/src/components.Bootstrap(0x0?, 0x4000482200) ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/src/components/Kerberos.go:65 +0x398 ka-garage-south | created by main.main ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/main.go:88 +0x36c ka-garage-south | ka-garage-south | goroutine 50 [sleep]: ka-garage-south | time.Sleep(0x12a05f200) ka-garage-south | /usr/local/go/src/runtime/time.go:194 +0x124 ka-garage-south | github.com/kerberos-io/agent/machinery/src/components.ControlAgent.func1() ka-garage-south | /go/src/github.com/kerberos-io/agent/machinery/src/components/Kerberos.go:256 +0x54 ka-garage-south | created by github.com/kerberos-io/agent/machinery/src/components.ControlAgent

It looks like it crashed while try to restart. Interesting find!

instantdreams commented 1 year ago

I am glad I've gone through this discovery with you.

I am not sure why the polygon coordinates are not being saved.

I do also feel that I would likely bind a mount to a config location and manage the config file that way, but this was great discovery process.

cedricve commented 1 year ago

@instantdreams I've already added a possible fix https://github.com/kerberos-io/agent/commit/8507e84ea0c925d438cd28e2d8f158ae67194141. Referencing your issue above, I believe the decoder was already closed before the motion function was closed.

cedricve commented 1 year ago

Which browser are you using to draw the region of interest?

instantdreams commented 1 year ago

I was using Firefox. I also tried with Chrome with the same result - nothing was saved.

instantdreams commented 1 year ago

If using specific mount points in docker compose, the original issue is resolved using the following steps:

  1. Create mount points for configuration (if using a targeted mount for configuration)

    cd [location of docker-compose.yml]
mkdir [camera] [camera]/config
wget https://raw.githubusercontent.com/kerberos-io/agent/master/machinery/data/config/config.json -O [camera]/config
sudo chown [user running docker]:[group running docker] [location of docker-compose.yml] -R
sudo chmod 777 [location of docker-compose.yml] -R

  2. Create mount points for media

    mkdir [media location]/[camera] [media location]/[camera]/recordings [media location]/[camera]/snapshots
sudo chown [user running docker]:[group running docker]  [media location] -R
sudo chmod 777 [media location]/[camera] -R

  3. Use the correct targets in the volumes section of docker-compose.yml

    volumes:
  - [location of docker-compose.yml]/[camera]/config:/home/agent/data/config:rw
  - [media location]/[camera]/recordings:/home/agent/data/recordings:rw
  - [media location]/[camera]/snapshots:/home/agent/data/snapshots:rw

This will resolve the "permission denied" error caused by the lack of a specified "snapshots" volume.

Thank you to @cedricve for confirming this.

cedricve commented 1 year ago

Thanks for reporting. Does that help saving the polygon coordinates?

instantdreams commented 1 year ago

It does not resolve the polygon issue, which I think is unrelated. I have created a new issue #63 to capture that problem.