This is the first of a two part series to pass all of ActiveRecord's MySQL test cases. To do this we need to have a test harness sytem and minimal amount of code to load the ActiveRecord test schema(s). The quest to full-stack Lambda for all is close!
My TinyTDS gem for FreeTDS is heavily inspired by the Mysql2 library. Both's interface pattern relies on two key objects, a Client and Result. To ensure our adapter requires the least work possible, we need to replace all Mysql2 adapter interfaces (the @connection object) with a facade backed by the Aws::RDSDataService::Client.
ActiveRecord Test Harness
Most 3rd party adapter gems (Oracle, SQLServer, etc) like to run the full ActiveRecord test suite along with their own unit or integration tests. This is kind of tricky but ActiveRecord does a really good job supporting it and I copied a lot of code I wrote in the SQL Server adapter over to this gem. Highlights:
Gemfile addition allows Rails git repo to be bundled.
Allow RAILS_VERSION env to target different tags.
Test support files to resolve local & Rails git repo test paths.
Sets the ARCONFIG env for our adapter's config file.
Target test via TEST_FILES (ours) TEST_FILES_AR (activerecord) env vars.
Aurora Serverless Test DBs
In our initial work via (#2) we used CDK to create our Aurora Serverless test database. During this work I discovered two major chagnes were needed.
Create two Aurora Serverless clusters since each can only have one database. One for activerecord_unittest and one for activerecord_unittest2. Both are used by ActiveRecord's test suite.
Support emoji (🎄🎅) utf8mb4 and large prefixes by creating a new DB Cluster Parameter Group using the aurora5.6 family. This cluster uses paramaters like innodb_file_format: "Barracuda" and misc character encoding settings.
Native Gems?
The Aws::RDSDataService::Client uses AWS HTTP client code to talk to Aurora Serverless via the Data API. This requires no native gems to use. However, our adapter does inherit from the core Mysql2Adapter and doing so requires this code to be run.
gem "mysql2", ">= 0.4.4"
require "mysql2"
To avoid requiring and building code we do not need. This project includes a small Rubygems hack which keeps the Mysql2 gem from being really loaded.
Current Test Status?
I have not run the full ActiveRecord test suite. For now I have setup the bin/test file to only run one test in ActiveRecord's adapter_test.rb case. The goal is to ensure we are loading the schema successfully which happens prior to the first test. Which we have done 🎉
However, my first attempt at this whole file we passed most of the tests. So I suspect Phase II is going to go really smoothly and I bet we are passing ~75% of ActiveRecord's tests now! Other details include:
PostgreSQL?
Aurora Serverless does support PostgreSQL-compatible editions. However, I have no interest in hacking two native gem adapters now or likely in the future. I have however done a lot of work to allow this to easily happen if some person with a lot of gumption decided to contribute to this adapter. Here are a few points if you are interested in the topic.
Search this code base for TODO with the tag [PG] for notes.
Please make a aurora_serverless/postgresql directory for all code.
Put all client code into a module. Mix into AuroraServerless::Client.
Make sure we conditionally set ARCONFIG to a PG-specific file.
Goal
This is the first of a two part series to pass all of ActiveRecord's MySQL test cases. To do this we need to have a test harness sytem and minimal amount of code to load the ActiveRecord test schema(s). The quest to full-stack Lambda for all is close!
Prior Art & Architecture
Thankfully due to my previous ActiveRecord adapter experience I had two open source projects to pull from.
Aws::RDSDataService::Client Facade
My TinyTDS gem for FreeTDS is heavily inspired by the Mysql2 library. Both's interface pattern relies on two key objects, a
ClientandResult. To ensure our adapter requires the least work possible, we need to replace all Mysql2 adapter interfaces (the@connectionobject) with a facade backed by the Aws::RDSDataService::Client.ActiveRecord Test Harness
Most 3rd party adapter gems (Oracle, SQLServer, etc) like to run the full ActiveRecord test suite along with their own unit or integration tests. This is kind of tricky but ActiveRecord does a really good job supporting it and I copied a lot of code I wrote in the SQL Server adapter over to this gem. Highlights:
RAILS_VERSIONenv to target different tags.ARCONFIGenv for our adapter's config file.TEST_FILES(ours)TEST_FILES_AR(activerecord) env vars.Aurora Serverless Test DBs
In our initial work via (#2) we used CDK to create our Aurora Serverless test database. During this work I discovered two major chagnes were needed.
activerecord_unittestand one foractiverecord_unittest2. Both are used by ActiveRecord's test suite.utf8mb4and large prefixes by creating a new DB Cluster Parameter Group using theaurora5.6family. This cluster uses paramaters likeinnodb_file_format: "Barracuda"and misc character encoding settings.Native Gems?
The
Aws::RDSDataService::Clientuses AWS HTTP client code to talk to Aurora Serverless via the Data API. This requires no native gems to use. However, our adapter does inherit from the coreMysql2Adapterand doing so requires this code to be run.To avoid requiring and building code we do not need. This project includes a small Rubygems hack which keeps the
Mysql2gem from being really loaded.Current Test Status?
I have not run the full ActiveRecord test suite. For now I have setup the
bin/testfile to only run one test in ActiveRecord'sadapter_test.rbcase. The goal is to ensure we are loading the schema successfully which happens prior to the first test. Which we have done 🎉However, my first attempt at this whole file we passed most of the tests. So I suspect Phase II is going to go really smoothly and I bet we are passing ~75% of ActiveRecord's tests now! Other details include:
PostgreSQL?
Aurora Serverless does support PostgreSQL-compatible editions. However, I have no interest in hacking two native gem adapters now or likely in the future. I have however done a lot of work to allow this to easily happen if some person with a lot of gumption decided to contribute to this adapter. Here are a few points if you are interested in the topic.
TODOwith the tag[PG]for notes.aurora_serverless/postgresqldirectory for all code.AuroraServerless::Client.ARCONFIGto a PG-specific file.