neo4jrb / activegraph

An active model wrapper for the Neo4j Graph Database for Ruby.
http://neo4jrb.io
MIT License
1.4k stars 277 forks source link

Migration check affects rails performance in test and production #1655

Open yourpalal opened 3 years ago

yourpalal commented 3 years ago

When used in rails, a middleware is added to the rack server which negatively affects performance.

CheckPending performance

On every rails request, including in production, the list of migrations is found via ActiveGraph::Migrations::Runner#latest_migration. This method transitively calls the files method, which checks the filesystem for migration files and parses the name of each migration.

In a production setting, the actual pending migrations check will only happen the first time the middleware runs, but the performance hit of calling latest_migration will be added to every request.

I noticed the performance hit of this call while profiling a test suit with stackprof flamegraphs. It seems to take more time than many queries! By instrumenting the call to latest_migration and measuring it with stackprof, I found that it accounted for 3.5% of the duration of one of our api request spec files. Each call takes approximately 5-7ms on my machine - longer than most sql or neo4j queries.

Configuration

Pending migrations checks are configured according to two configuration options:

What could be improved

Rails.env == 'test'

In a rails test environment, skip_migration_check is set to nil (false), meaning no error will be raised, but fail_on_pending_migrations is set to true, meaning the middleware will be added anyway!

An improvement for this case would be to set fail_on_pending_migrations to false in rails test environments, or whenever skip_migration_check is set to false. There's really no need to run the middleware when it's never going to raise an error.

This will speed up rails test suites a bit, which is nice!

Rails.env == 'production'

Another improvement could be to modify the checkpending middleware to avoid repeated latest_migration calls in production. This could be done by either:

Setting config.neo4j.fail_on_pending_migrations to false will allow you to avoid the middleware altogether. This does however mean that you lose the protection it provides