[Support]: Env-Var substutition does not work for MQTT user/password

kquinsland commented 1 year ago

Describe the problem you are having

I can not get a successful connection to my MQTT broker when using env-vars in the MQTT broker configuration.

mqtt:
  host: mqtt.internal

  # Test 1: DOES NOT WORK - Invalid auth
  #user: '{FRIGATE_MQTT_USER}'
  #password: '{FRIGATE_MQTT_PASSWORD}'

  # Test 2:  DOES NOT WORK - Invalid Auth
  #user: "{FRIGATE_MQTT_USER}"
  #password: "{FRIGATE_MQTT_PASSWORD}"

  # Test 3: DOES NOT WORK - validation failure
  #user: {FRIGATE_MQTT_USER}
  #password: {FRIGATE_MQTT_PASSWORD}

  # Test 4: DOES WORK - Note: I have rotated the credentials since posting
  user: "frigate"
  password: "psalms.latter.dross.mabel.argument.aster.greasy.westerly.assai.tobit.FORKLIFT.does.tarn"

cameras:
  test:
    ffmpeg:
      inputs:
        # See below, I expect this to be incorrect
        - path: rtsp://{FRIGATE_MQTT_USER}:{FRIGATE_MQTT_PASSWORD}@test-cam.internal:554/cam/realmonitor?channel=1&subtype=0

Obviously, I changed the user/pass for my MQTT broker since posting this however the credentials in the configuration below are the ones that I was attempting to use / testing with. I did not modify them / replace them with generic myPasswordHere values in the hopes of others being able to reproduce the issue.

When the first or second test case is active, this is what I see in the logs:

frigate.mqtt                   DEBUG   : MQTT connected
frigate.mqtt                   ERROR   : Unable to connect to MQTT server: MQTT Not authorized
frigate.mqtt                   DEBUG   : MQTT connected
frigate.mqtt                   ERROR   : Unable to connect to MQTT server: MQTT Not authorized
frigate.mqtt                   DEBUG   : MQTT connected

this will loop over and over until I stop the container.

When the credentials come from the third test case, pydantic gets upset:

]: [2023-03-04 16:38:35] frigate.app                    INFO    : Starting Frigate (0.11.1-2eada21)
]: *************************************************************
]: *************************************************************
]: ***    Your config file is not valid!                     ***
]: ***    Please check the docs at                           ***
]: ***    https://docs.frigate.video/configuration/index     ***
]: *************************************************************
]: *************************************************************
]: ***    Config Validation Errors                           ***
]: *************************************************************
]: 'user'
]: Traceback (most recent call last):
]:   File "/opt/frigate/frigate/app.py", line 332, in start
]:     self.init_config()
]:   File "/opt/frigate/frigate/app.py", line 82, in init_config
]:     user_config = FrigateConfig.parse_file(config_file)
]:   File "/opt/frigate/frigate/config.py", line 942, in parse_file
]:     return cls.parse_obj(config)
]:   File "pydantic/main.py", line 521, in pydantic.main.BaseModel.parse_obj
]:   File "pydantic/main.py", line 339, in pydantic.main.BaseModel.__init__
]:   File "pydantic/main.py", line 1056, in pydantic.main.validate_model
]:   File "pydantic/fields.py", line 857, in pydantic.fields.ModelField.validate
]:   File "pydantic/fields.py", line 1074, in pydantic.fields.ModelField._validate_singleton
]:   File "pydantic/fields.py", line 1121, in pydantic.fields.ModelField._apply_validators
]:   File "pydantic/class_validators.py", line 313, in pydantic.class_validators._generic_validator_basic.lambda12
]:   File "pydantic/main.py", line 704, in pydantic.main.BaseModel.validate
]:   File "pydantic/main.py", line 339, in pydantic.main.BaseModel.__init__
]:   File "pydantic/main.py", line 1056, in pydantic.main.validate_model
]:   File "pydantic/fields.py", line 840, in pydantic.fields.ModelField.validate
]:   File "pydantic/fields.py", line 1121, in pydantic.fields.ModelField._apply_validators
]:   File "pydantic/class_validators.py", line 282, in pydantic.class_validators._generic_validator_cls.lambda4
]:   File "/opt/frigate/frigate/config.py", line 66, in validate_password
]:     if (v is None) != (values["user"] is None):
]: KeyError: 'user'
]: *************************************************************
]: ***    End Config Validation Errors                       ***
]: *************************************************************
]: [cmd] python3 exited 1
]: [cont-finish.d] executing container finish scripts...
]: [cont-finish.d] done.
]: [s6-finish] waiting for services.
]: [s6-finish] sending all processes the TERM signal.
]: [s6-finish] sending all processes the KILL signal and exiting.

The 4th test case is me hard-coding the credentials right into the yml file. There are no issues here; the logs do not look like they do with any of the three prior test cases.

The camera portion of the config.yml provided is intentionally incorrect and - as expected - produces errors like this in the logs:

]: [2023-03-04 16:41:18] ffmpeg.test.detect       ERROR   : [rtsp @ 0x55fa72061000] method OPTIONS failed: 401 Unauthorized
]: [2023-03-04 16:41:18] ffmpeg.test.detect       ERROR   : rtsp://frigate:psalms.latter.dross.mabel.argument.aster.greasy.westerly.assai.tobit.FORKLIFT.does.tarn@test-cam.internal:554/cam/realmonitor?channel=1&subtype=0: Server returned 401 Unauthorized (authorization failed)
]: [2023-03-04 16:41:18] frigate.video                  ERROR   : test: Unable to read frames from ffmpeg process.

The camera should return a 401 here as the credentials contained in the FRIGATE_MQTT_USER and FRIGATE_MQTT_PASSWORD vars are incorrect for the camera. The reason why I used the mqtt env-vars is because the "resolved" values are emitted via logs which allowed me to confirm that the values that {FRIGATE_MQTT_PASSWORD} and {FRIGATE_MQTT_USER} resolve to are correct.

I can copy the user/pass from the "server returned 401" log line and they match the user/pass in my password manager. Additionally, I can copy the user/pass straight from the log line and paste them into my MQTT client ... and get in.

Furthermore, when I change the env-vars in the rtsp://.... expression back to the correct values, frigate is able to connect to the camera and authenticate as expected.

I am running frigate via docker run ... with an env-vars file:

docker run --rm --name frigate \
    --env-file /etc/frigate/secrets \
     <...>
    blakeblackshear/frigate:stable

Version

0.11.1-2eada21

Frigate config file

(see main post body)

Relevant log output

(see main post body)

FFprobe output from your camera

N/A

Frigate stats

No response

Operating system

Other Linux

Install method

Docker CLI

Coral version

M.2

Network connection

Wired

Camera make and model

N/A

Any other information that may be helpful

No response

NickM-27 commented 1 year ago

MQTT user substation is only supported in 0.12 so that's why it fails.

kquinsland commented 1 year ago

MQTT user substation is only supported in 0.12 so that's why it fails.

That ... makes sense.

Small PR that would have saved me some time: https://github.com/blakeblackshear/frigate/pull/5641

blakeblackshear / frigate