Add retry feature for http secret-store plugin

[YoonSungHyun-Git]

Use Case

Use the scretstore http plugin to retrieve connection information from postgresql output. However, in the current situation, if the secretstore http plugin fails to retrieve the API value, the service does not start because @ variable substitution is not performed.

Expected behavior

• API call setup : Use @ variable
• API connection failure : Retry

Actual behavior

• API call setup : Use @ variable
• API connection failure : service down ( The reason is that the @ variable does not have a value and attempts to connect in an unsubstituted state.)

Additional info

[[secretstores.http]]
  id = "meta"
  url = "http://localhost/meta"
  timeout = "5s"

[[outputs.postgresql]]
  connection = "host=@{meta:host} port=@{meta:port} user=@{meta:user} password=@{meta:password} dbname=@{meta:database} sslmode=disable"

[srebhan]

Next steps: Look into code on how to realize this best. We probably cannot error out on startup then and all downstream users of the secret-store need to cope with the fallout. This might take some time...