Rewrite table names in an `Ecto.Query`

[venkatd]

Hi @TurtleAI/eds

If I want to update the table for an Ecto.Schema, I can do something like this:

source = Ecto.get_meta(record, :source)
record
|> Ecto.put_meta(source: "#{dynamic_prefix}_#{source}")

Is there something that similar that can be done to an Ecto.Query?

Something like

query
|> update_sources(fn source -> "#{dynamic_prefix}_#{source}" end)

Thanks!

[josevalim]

Unfortunately there is no way to do so after the query is built. You can only specify the source when you build the query in from. :/

Is there a reason why the value can’t be specified upfront?

[venkatd]

@josevalim

Good to hear from you! I saw on Twitter that you had an injury. I hope you're better.

If we specify the source when building the query, many queries in our app would have to accept an additional argument to allow them to be used in different reducers. And we have a lot.

Since we already commit all operations inside a single function called commit_operations I thought it would be more convenient to perform some kind of rewrite in that function. So we can handle it generically.

I had considered handling namespacing with postgres schemas and put_query_prefix, but I think it would add other complexities to our setup vs. table prefixes.

I noticed that %Ecto.Query{} has a sources field. Is there some workaround such as modifying the data structure directly? if so, what fields would need to be modified? Would doing this be non-trivial?

Here's an example:

defmodule Turtle.Cards.Reducer do
  alias Turtle.Cards.Models.Card
  alias Turtle.Event.{CardCreated, CardRenamed}

  # the namespace can make sure all operations (Ecto.Multi and Ecto.Query update statements)
  # prefix the table name with "test_cards"
  use Derive.EctoReducer, namespace: "test_cards", repo: Turtle.Repo, models: [Card]

  def partition(%{root_card_id: root_card_id}), do: root_card_id

  def handle_event(%CardCreated{card_id: card_id, attrs: attrs}) do
    insert(%Card{
      id: card_id,
      name: Map.get(attrs, :name)
    })
  end

  def handle_event(%CardRenamed{card_id: card_id, card_name: card_name}) do
    update({Card, card_id}, name: card_name)
  end

  def handle_event(%Turtle.Event.CardMoved{
        id: id,
        card_id: card_id,
        relation: relation,
        target_id: target_id
      }) do
    [
      # this is an `Ecto.Query` used in other places in our app
      Turtle.Project.CardMoveQuery.move({card_id, relation, target_id}), 
      update({Turtle.Project.Card, card_id}, version: id)
    ]
  end

  def handle_event(_event), do: nl
end

[josevalim]

If you want, you can change the query.from.source information directly (but only the binary field, you can't change the schema). You would be messing up with the internals but it is unlikely to change anyway, because if we change this, we would break all adapters (we say Ecto.Query is part of our "Plugin API" that may change between minor versions). But other than that, there is no official API.

Good to hear from you! I saw on Twitter that you had an injury. I hope you're better.

Thank you, it is getting better. :)

[venkatd]

but only the binary field, you can't change the schema

You mean for a tuple like {"custom_users_table", User}, I can only change the binary which refers to the table name right? If so, that's totally fine because the table is the only thing we want to control. The schema would remain the same in all circumstances.

The risk of the internals changing is OK for me. Our tests would catch it pretty quickly and our logic will be centralized to a single function. So it wouldn't cause us any major pain.

Thank you, it is getting better. :)

Great! I hope the forced time off came with some benefits too...

[josevalim]

You mean for a tuple like {"custom_users_table", User}, I can only change the binary which refers to the table name right?

exactly :)

Great! I hope the forced time off came with some benefits too...

learning how to slow down a bit!