Running on many apps in an umbrella

[iacobson]

Hi,

I am using the library in one application inside an umbrella with many other apps. Now I was trying to use it in another app. Each of the apps has its own repo and database.

Encountered 2 issues:

• the Rihanna.Supervisor is already started by one of the apps, so it cannot be started by the other one as well
• the configuration gets overwritten as part of the umbrella config

Would it be a way to make it work in an umbrella app?

Thanks

PS: isn't that funny? having trouble running rihanna under the umbrella

[samsondav]

@iacobson Haha love the pun.

Actually it's quite possible to run Rihanna as part of an umbrella app, and I have run it like this before. You'll need to boot the dispatcher separately and pass in the configuration at runtime.

Check out the dispatcher here: https://github.com/samphilipd/rihanna/blob/master/lib/rihanna/job_dispatcher.ex

If you have any more questions lmk

[iacobson]

Thanks, would that mean to implement a different Supervisor?

So my current config is:

• I do not have this at all in the config, because of umbrella apps common config:

  config :rihanna,
  producer_postgres_connection: {Ecto, MyApp.Repo}

• in my application file:

    {Rihanna.Supervisor, [postgrex: MyApp1.Repo.config()]},

  same for MyApp2

The error I get when starting the server:

** (Mix) Could not start application my_app_1: MyApp1.start(:normal, []) returned an error: shutdown: failed to start child: Rihanna.Supervisor
    ** (EXIT) shutdown: failed to start child: Rihanna.Job.Postgrex
        ** (EXIT) already started: #PID<0.1095.0>

The same would happen with Rihanna.TaskSupervisor or any other named process.

Would you have some example from an umbrella app?

[samsondav]

Yup!

We have one dedicated umbrella app for the worker:

defmodule RihannaWorker.Application do
  @moduledoc false

  use Application

  def start(_type, _args) do
    db = Keyword.take(MyApp.Repo.config(), [:username, :password, :database, :hostname, :port])

    children = [
      {Task.Supervisor, name: Rihanna.TaskSupervisor},
      %{
        id: Rihanna.JobDispatcher,
        start: {Rihanna.JobDispatcher, :start_link, [[db: db], [name: Rihanna.JobDispatcher]]}
      }
    ]

    opts = [strategy: :one_for_one, name: RihannaWorker.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

You can enqueue jobs from elsewhere simply with Rihanna.enqueue.

[iacobson]

Thanks, I created a dedicated umbrella app for Rihanna. This partially solves the issue. I can start the Task.Supervisor there, but not the JobDispatcher.

I have different Repos for different umbrella apps where using Rihanna.

So what I did, I started the TaskSupervisor from the new RihannaWorker app, and started a dispatcher in the application.ex of each app where using Rihanna like:

def start(_type, _args) do
    db_config =
      Keyword.take(MyApp1.Repo.config(), [:username, :password, :database, :hostname, :port])

children = [

...
      %{
        id: Rihanna.MyApp1JobDispatcher,
        start:
          {Rihanna.JobDispatcher, :start_link,
           [[db: db_config], [name: Rihanna.MyApp1JobDispatcher]]}
      }

...
end

Same for MyApp2 etc.

But now I have a new problem: Rihanna.Job.Postgrex As I could not add the config :rihanna, :producer_postgres_connection to each app because of the umbrella, it needs the default Rihanna.Job.Postgrex that I cannot configure for each app.

Just for testing purposes, I ended up with a hack: dynamically setting the config when enqueueing the job, like:

Application.put_env(:rihanna, :producer_postgres_connection, {Ecto, MyApp1.Repo})
Rihanna.enqueue(MyApp1.SomeTask, [])

But I do not feel confident in this to use it as a solution.

Any recommendations?

[samsondav]

Uff Yes that's not a good solution.

Probably the only way to resolve this is to allow an optional argument to Rihanna.enqueue that takes a connection - which can be wrapped for each app.

Would you like to put together a PR to do this?

[iacobson]

I added a WIP PR with some observations and questions. Once we clarify those, I'm happy to continue working on this https://github.com/samphilipd/rihanna/pull/82