Strech / avrora

A convenient Elixir library to work with Avro messages, schemas and Confluent® Schema Registry
https://hexdocs.pm/avrora
MIT License
97 stars 33 forks source link

Any way to configure the registry_url at startup? #90

Closed juanperi closed 3 years ago

juanperi commented 3 years ago

We are currently using

  use Avrora.Client,
    registry_url: Application.get_env(:the_app, TheApp.AvroClient)[:url],
    schemas_path: Application.get_env(:the_app,  TheApp.AvroClient)[:schemas_path]

to define our own client module. But as you already know, those configurations must come from compile time environment. Is there any way to configure it at runtime, when the application starts? Tried looking at the supervisor tree, but didn't find anything.

Thank you in advance for your answer, and thank you for your awesome library!

juanperi commented 3 years ago

btw, I saw other issues dealing with compile time configuration and fixes. But I haven't seen anything related to fully runtime (startup) configuration

Strech commented 3 years ago

Hey @juanperi seems like there are some demand on runtime 😆

Since we have new configuration option otp_app: :name I think we can extend its usage and take most of the settings if they set from the OTP app and if not – fallback to default value.

Then the chain would be

  1. If otp_app is set and schemas_path is set – apply Application.app_dir
  2. If otp_app is set and schemas_path is not – take Application.get_env(<otp_app>, __MODULE__, :schemas_path)
  3. If otp_app is not set and schemas_path is set – take schemas_path
  4. If otp_app is not set and schemas_path is not set – take default value

And the same-ish logic for rest options, WDYT?

juanperi commented 3 years ago

that would still allow for 1 client for otp app, right? What if instead we allow options to be passed when the start_link is called on the CustomAvroClient?

      children = [
        {TheApp.AvroClient, [schemas_path: xxxx, registry_url: yyyyy]}
      ]

      {:ok, pid} = Supervisor.start_link(children, strategy: :one_for_one)

That way, the initialization of the process will be able to pass the config, and make it fully independent of any compile time dependency, and allow you to have as many configurations as you need.

The usecase i have, is that I build a single docker image for QA and Prod. I need to point the client to point to QA when started in QA mode, and in production mode to a different schema.

The otp app might work in this case, but isn't it more straightforward and flexible to just pass the parameters when starting the supervision tree? The developer is in charge of where that config is coming from.

Strech commented 3 years ago

Yeah, initially the supervision tree args were complicated due to the Private client, but I guess I can take a look again into it. So will try that way

juanperi commented 3 years ago

can you expand a bit in the complication?

Strech commented 3 years ago

Also with Application.get_env I've suggested you still have control over environment, since you will be able to do it this way

# config/test.exs
config :the_app, TheApp.AvroClient, 
  registry_url: "http://xxx.xx"
defmodule TheApp.AvroClient do
  use Avrora.Client,
    otp_app: :the_app
end

because the resolution will happen in runtime, you can have different config per environment. I think it should work for you.

P.S Complications there because Avrora is not running Avrora.Config as a process to hold the config from supervision setup

juanperi commented 3 years ago

you are totally correct. that would work if you automatically scope the config to the current client as in your last example, we could have multiple clients, with different configurations. That would be good enough for having custom configurations per client

Strech commented 3 years ago

Yes, configuration will be scoped to the client, like this get_env(:the_app, __MODULE__, :registry_url) and __MODULE__ will be resolved per-client.

I will open PR and mention you there 👍🏼