search

cockroachdb / activerecord-cockroachdb-adapter

CockroachDB adapter for ActiveRecord.
Apache License 2.0
103 stars 52 forks source link

[Tracking] Outstanding ActiveRecord 5.2 tests #48

Open alimi opened 4 years ago

alimi commented 4 years ago

Below is a list of all the ActiveRecord tests that get run against PostgreSQL on Rails 5.2. Check off tests once they're passing against master.

There will be tests that don't work against CockroachDB. To skip these tests

  1. Add a ruby file to test/excludes that matches the name of the test class if one doesn't already exist. For example to exclude a test from ActiveRecord::AdapterTest, create test/excludes/ActiveRecord/AdapterTest.rb.
  2. Add an exclude statement to the file with the name of the test to exclude and a description. For example to exclude test_indexes from ActiveRecord::AdapterTest: https://github.com/cockroachdb/activerecord-cockroachdb-adapter/blob/c3cb637bd855b36bbf9e76b7947162054b95022e/test/excludes/ActiveRecord/AdapterTest.rb#L1
  3. Finally, if the test can run against CockroachDB with a few changes, add it to test/cases. Everything from the ActiveRecord test suite will be available, so a lot of the excluded test can be copied over. Namespace the test under the CockroachDB module to avoid name collisions. See test/cases/adapter_test.rb for an example.

For information on running tests, see the Contributing guide.

ActiveRecord tests

rafiss commented 4 years ago

Thanks for compiling this! I did a quick scan of the tests here, and a few jumped out to me as things that are pretty likely not to work, based on what CockroachDB supports right now:

alimi commented 4 years ago

After a few PRs get merged, we'll be down to 60 test files to get through. Thanks to @jordanlewis's tip to skip load_schema after it's been run once, we can run most tests much faster.

It can take anywhere from 1 to 4 hours to run a test file, figure out why it's failing, and apply a fix or skip the tests. With 60 outstanding test files, we're looking at between 60 and 240 hours of work. The last 10 files have come in at an hour or less, and if that continues we’ll be closer to 60 hours.

To help speed this up a bit, people could quickly scan the tests like @rafiss did (https://github.com/cockroachdb/activerecord-cockroachdb-adapter/issues/48#issuecomment-609489397) and pick out anything that we know isn't going to work with CockroachDB. People are also welcome to jump in and investigate the failing tests.

rafiss commented 4 years ago

I took a rough stab at trying to prioritize the remaining test files. This is very unscientific, but I think it still could be a helpful heuristic to figure out which things to tackle next with our remaining time. I did my best, but it's still quite possible that some of the ones that I've marked as higher priority won't be able to work with CockroachDB, so as always, if you hit a stumbling block, let us know, so we can determine if the test should just be ignored.

High priority

Medium priority

Low priority

May not work with cockroachdb

marlabrizel commented 4 years ago

I don't have edit permissions on the issue, but test/cases/primary_keys_test.rb passes for me against AR. I think this one may be safe to mark as completed unless I'm missing something!

rafiss commented 4 years ago

Sounds good! I was just basing this off of the boxes that are currently unchecked in Ali's issue description above.

alimi commented 4 years ago

I created individual issues for the outstanding high/medium/low priority test files. I included the failures/errors to make it easier to see what's wrong in the tests. It might make sense to re-prioritize based on the failures/errors.

You can find the issues here: https://github.com/cockroachdb/activerecord-cockroachdb-adapter/issues?q=is%3Aissue+is%3Aopen+%5BActiveRecord+5.2+Test+Failure%5D.

There are a few test files that are no longer failing, and I didn't create issues for them.

It looks like most of the "May not work with cockroachdb" tests don't work with CockroachDB. I didn't create issues for them either.