Add `aws_aurora`

[dfellis]

Digging into this task:

1. Aurora is treated by the SDK as just another feature of RDS (though with wildly non-overlapping configuration knobs). Whether we should break this out into a separate aws_aurora module or make it part of aws_rds is something we need to think about.
2. There's an Aurora v1 and v2, with v1 only available in certain regions with fewer features and having some special flags just for itself. It also seems Aurora v1 MySQL is deprecated at the end of February next year while Aurora v1 Postgres was deprecated at the end of January this year while another batch of deprecation (of old Postgres versions on Aurora v2) is happening February next year.

I think we should not support Aurora v1 at all, but I am still not sure if it should be a separate module or not.

Relatedly, some of the complexity here is that Aurora is intertwined with multi-region RDS, so like aws_dynamo it seems unlikely that we can fully implement this module until we refactor everything for multi-region support.

[aguillenv]

Using the AWS console, aurora instances are listed as RDS instances or are they different resources? I think the answer to that question is the answer if they should be different modules or not. We have other modules that use the same SDK API behind scenes like aws_vpc, aws_security_groups or aws_ec2 all using AWS EC2 API.

I agree we should only support v2.

[dfellis]

So, Amazon has Aurora inside of RDS in the dashboard as a special database type with two configurations. It also automatically requires you to name the "DB cluster" instead of the DB instance like the others, but that's subtle, so that implies this task should go into the aws_rds module.

The only thing that's got me still considering an alternative is the significant lack of overlap between Aurora configuration properties and "normal database" configuration properties. Their API is ungainly the way it is structured and having a ton of NULL configuration columns for users (whether they're using Aurora or not) feels bad to me.

Possibly it could be a special aurora table inside of the aws_rds module, but then why require "regular" RDS users to deal with it?

[dfellis]

Why Aurora is not given top-level billing while Dynamo is makes no sense to me. AWS's marketing is totally inconsistent, just like their APIs, but we wanted to keep our groupings similar to how AWS themselves do it so we have less to explain to users, so putting in new aurora tables (I'm thinking aurora_cluster and aurora_instance with the latter being not required if they chose serverless aurora) is how it should be structured. I guess?

[depombo]

Given all the information you provided here, I agree with your conclusion to make aurora tables within rds

[aguillenv]

Taking a look at the console, in the end, aurora is not just any other rds db engine? I'm doubting why we need a module or special table for it. I understand that it requires a cluster, and taking a look I see that MySQL too. Why not add a cluster FK to the current rds instance table and make a constraint that the cluster cannot be null for aurora or MySQL? Or does aurora have special API calls?

[yrobla]

I agree, i see conceptually as another RDS type, maybe with more specific configs... but at the end i think it should be under RDS

[dfellis]

So if you read through the CreateDBClusterCommandInput and CreateDBInstanceCommandInput properties, you quickly see that there is almost zero overlap between Aurora configuration and non-Aurora configuration. There are 47 mentions of "Aurora" in the CreateDBInstanceCommandInput page and most of them are saying "this property doesn't apply to Aurora" or "this property is only for Aurora."

In comparison, the other DB types only rarely have special properties, like this one which is the only special property for Microsoft SQL Server which is pretty crazy to me since it runs on a different operating system from all of the other databases.

Aurora is also the only one that has configurable "database modes" to mimic MySQL 5.6, 5.7/8.0 (wut?), or "Postgres" (no explicit version specified), and does not support explicit versioning, which makes engineVersion suddenly a nullable field.

Basically, if I treat Aurora as just another database type, it significantly impacts the usability of both Aurora and non-Aurora database types, since we can't hide columns from the user.

My personal preference would be to make aurora a separate package, perhaps named aws_rds_aurora so it's clear it's related to RDS but this feature isolated from the rest, because of the cognitive overhead of managing Aurora versus the other DBs. But I am okay with it being a separate set of tables inside of the aws_rds module, as well.

I don't like the idea of it being mixed in with the others, though. It's basically shoehorned into RDS, afaict, and isn't a good fit.

[aguillenv]

ohh I see, then if cluster command and instance commands are different for aurora in general, I'd lean towards the aws_rds_aurora module. Also, we should add a filter in current aws_rds to not load aurora instances if it is doing that now.