ability to conditionally update capacity

[michaelglass]

We have a rails application that uses dynamo. It's useful to have table definitions as migrations to ensure developers have up-to-date schemas. However, changing capacity is something that belongs in configuration management (e.g. terraform) rather than in my application code.

One solution is to have a flag in TableConfig that makes says, "only set capacity on table creation". Slowly working through that in this branch: https://github.com/NoRedInk/aws-sdk-ruby-record/tree/optionally-update-throughput

[awood45]

Can you elaborate a bit more about what you're trying to do here? If the TableConfig has a different capacity after table creation you don't want it to change?

In any case, I don't think remote configuration code like a capacity flag should live in the model, like I see in your first commit. The idea is to achieve a full separation.

[michaelglass]

So we have our application, which contains database structure & migrations Then we have terraform, which contains infrastructure shape & capacity

I'd like to create databases & change database shape without changing capacity. We need some kind of initial capacity, but I'd like to separate the idea of an "initial capacity" from a "current capacity", and then move the "current capacity" from our application code to our configuration code.

Does that make sense?

[awood45]

Okay, I think I see what you're getting at now - you want a higher level abstraction of piecemeal migrations, rather than a declarative config?

To get an idea of what we would want to build, here is how you'd do that now.

class Model
  include Aws::Record
  # model specific attribute and index definitions go here
end

@initial_capacity = Aws::Record::TableConfig.define do |t|
  t.model_class Model
  t.read_capacity_units 1
  t.write_capacity_units 1
end

@current_capacity = Aws::Record::TableConfig.define do |t|
  t.model_class Model
  t.read_capacity_units 5
  t.write_capacity_units 5
end

# notably, you can have a method like this
def alter_capacity(read:,write:)
  @current_capacity.read_capacity_units(read)
  @current_capacity.write_capacity_units(write)
  @current_capacity.migrate!
end

I can see where this could fall short depending on your use case, can you possibly give a small example of how what you're proposing might work?

[michaelglass]

yup, that's what we do right now. I feel like that's going to be a common pattern.

[awood45]

It accomplishes the goal of separating the model from the remote configuration, but I'm open to feature requests and alterations if we can hash out where this is causing pain. I don't yet understand what we're trying to do differently - does it have to do with Terraform integration points/pain? If so I could try to look in to Terraform a bit more and see if there's a way to smooth that.

[michaelglass]

If I think about using Postgres with, for example, rails:

• Managing, permuting the schema: most "rails" developers via migrations.
• Postgres deployment, maintenance, performance monitoring, availability etc. is the responsibility of (often a subset of those same developers) who have fluency with monitoring, configuration management, etc.

It's not about specific pain -- e.g. we can do it this way. But when I saw the code in our application, I thought, "this feels like the common case and should be a library feature".

Anyways, happy to do the work if it's interesting / if we can agree upon an API.

[natesalisbury]

"only set capacity on table creation"

I would also like to be able to do this.

[awood45]

Would integration with the table auto-scaling functionality that has been released accomplish the end goal for this better? That's something in a similar vein we've been looking at now that the feature exists.

[natesalisbury]

I would still like the option to manage capacity settings from outside of the table config. I'm currently using auto-scaling but I have been updating those settings from the web console. That way I don't have to also remember to update the table config when I make changes.