Example of How to Start Children like a Regular Supervisor

[RodolfoSilva]

@derekkraan, is this the correct way to start workers with the application? I was wondering if there is a better approach to initiate some jobs and automatically restart these jobs if they crash.

If so, can I add this to the documentation?

application.ex

defmodule MyApp.Application do
  use Application
  def start(_type, _args) do
    children = [
      {Horde.DynamicSupervisor, [name: MyApp.DistributedSupervisor, strategy: :one_for_one]},
      MyApp.Supervisor
    ]

    Supervisor.start_link(children, strategy: :one_for_one)
  end
end

supervisor.ex

defmodule MyApp.Supervisor do
  use Supervisor
  require Logger

  def start_link(init_arg \\ []) do
    Supervisor.start_link(__MODULE__, init_arg, name: __MODULE__)
  end

  @impl true
  def init(_init_arg) do
    children =
      [
        {Task, [id: :task, args: [:infinity]]},
        {Agent, [id: :agent, args: [fn -> %{} end]]},
        {GenServer, [id: : gen_server, args: [DefinedGenServer, {500, pid}]]}
      ]

    children
    |> Enum.map(&create_child_spec/1)
    |> Supervisor.init(strategy: :one_for_one)
  end

  defp create_child_spec({worker, opts}) do
    %{
      id: worker,
      restart: :transient,
      start:
        {Task, :start_link,
         [
           fn ->
             child_spec =
               Enum.into(opts, %{
                 id: worker,
                 type: :worker,
                 restart: :transient,
                 start: {worker, :start_link, opts[:args] || []}
               })

             Logger.info("Starting worker #{inspect(worker)} over HordeSupervisor")

             Horde.DynamicSupervisor.start_child(MyApp.DistributedSupervisor, child_spec)
           end
         ]}
    }
  end

  defp create_child_spec(worker), do: create_child_spec({worker, []})
end

I've made this inspired in this test:

https://github.com/derekkraan/horde/blob/065ffd49e660aa88648cddf6294a675963fa3eea/test/nested_supervision_test.exs#L29-L46

[RodolfoSilva]

@derekkraan, would it be possible to add this feature to Horde, similar to what you did in Highlander?

[derekkraan]

Is there a reason to not just use Highlander in this situation? It is what I would recommend in any case.

[RodolfoSilva]

@derekkraan It seems that Highlander doesn’t rebalance or distribute processes across the cluster; it only ensures that a process will run on a single node and, if the process dies, brings it back up.

I’m not sure if Horde was designed for this purpose, but we observe that services are well-distributed across the cluster.

In my case, I'm using Broadway with RabbitMQ, and I need to have only one instance of Broadway running across the entire cluster. I have a few workflows with this limitation. Using Highlander, it seems they are spawned only on a single node.

Maybe I'm doing something wrong, and this could be achieved with Highlander...

[derekkraan]

Aha, I see what you are getting at now. Highlander does not indeed solve this use case. Horde can do it, but only with a little bit of hacking, since DynamicSupervisor was meant for dynamic children, not static ones. I would not call it a durable solution however.

I'm not sure if Horde was designed for this purpose

So actually it wasn't, although I understand why you want this.

[RodolfoSilva]

Do you have any suggestions on how we can achieve this behavior?

[derekkraan]

I unfortunately don't have any suggestions at this point in time. You could continue to use Horde like you are right now.

I will think about providing something to solve for this problem in HordePro. If you would be interested in beta testing that, you can sign up for the beta here.