Added an athena_example directory, which similar to mysql_example and pgsql_big_dedupe_example, runs the exact same flow but relying on Athena.
Athena (a managed Presto by AWS), is inherently different from MySQL and PostgreSQL, and adding support to Athena is quite a bit of a challenge. I've hidden most of the complexity in the athena_utils.py module.
Duo to Athena's parallel query execution behavior, the improvement is quite significant.
Added an athena_example directory, which similar to mysql_example and pgsql_big_dedupe_example, runs the exact same flow but relying on Athena. Athena (a managed Presto by AWS), is inherently different from MySQL and PostgreSQL, and adding support to Athena is quite a bit of a challenge. I've hidden most of the complexity in the athena_utils.py module. Duo to Athena's parallel query execution behavior, the improvement is quite significant.