Closed Lapizistik closed 3 years ago
Thanks for the detailed issue! I think the issue you're having is that rails test
will reset the entire test database after a run (hence why it works the first time, but not the second). You can see this because it is trying to TRUNCATE
each table to clear it. I'm not sure if this is new in rails 6, but you'll likely need to run the tests with a user that can modify the spatial_ref_sys
table.
Another option could be to find a way to ignore this table when resetting the database for tests, I'm not sure if this is possible off the top of my head, but I can look into it in a little bit.
@keithdoggett here is what we do, I think this addresses the need:
https://github.com/SpeciesFileGroup/taxonworks/blob/development/spec/spec_helper.rb#L103
Thanks a lot for your quick reply.
I understand that resetting the database causes the problem. AFAIK rails 5 should also do this and there the test works many times, so something is different here.
I found something else: If I replace
rails generate controller Example index # generate something to test
with
rails generate model Example name:string
I do not get any error on rails test
any more, so model tests are fine (and there the database also gets reset).
There is something weird going on with controller/view tests.
(So if someone knows where to ask on the rails side this would be helpful I think.)
Thanks a lot again!
Thanks @mjy for that! @Lapizistik were you able to try that suggestion? If that doesn't work can you take a look at the test logs for either version 5 or the model tests and see if the table cleaning commands are different than what you posted above? I'm confused about how model tests are working, but controller tests aren't so maybe this will help us see what's going on.
Thanks @mjy for that! @Lapizistik were you able to try that suggestion?
I did not try this (thanks to @mjy nevertheless!) as
I assume I could work around the bug by using DatabaseCleaner but I tried to stay on plain rails 6 for investigating the bug. (I'll provide the test logs soon.)
@Lapizistik Sorry for the confusion, indeed this is gem related code. I think we'd like to ultimately simplify in your direction so I'm curious where this will go.
And here the log files.
I wrote a small shell script to automatically setup the rails project and run the tests (it assumes that a postgis_template
exists, as described above): apa-test.zip (github does not like shellscripts, so I had to zip it)
I run it with different parameters:
./apa-test.sh rails6c >rails6c.log 2>&1
rails6c.log rails6c/log/test.log
TYPE=M ./apa-test.sh rails6m >rails6m.log 2>&1
rails6m.log rails6m/log/test.log
RAILS_VERSION=_5.2.5_ ./apa-test.sh rails5c >rails5c.log 2>&1
@Lapizistik thanks a lot for all this. I'll have some time soon to try to repro this and figure out what's going on. I looked through this gem and spatial_ref_sys
is only used in the database setup tasks and ignore_tables
for the SchemaDumper
.
These lines from rails6c.log
in the stacktrace are probably the root cause of the issue
/var/lib/gems/2.5.0/gems/activerecord-6.1.3.1/lib/active_record/connection_adapters/abstract/database_statements.rb:198:in `truncate_tables'
/var/lib/gems/2.5.0/gems/activerecord-6.1.3.1/lib/active_record/connection_adapters/abstract/query_cache.rb:22:in `truncate_tables'
/var/lib/gems/2.5.0/gems/activerecord-6.1.3.1/lib/active_record/tasks/database_tasks.rb:221:in `truncate_tables'
/var/lib/gems/2.5.0/gems/activerecord-6.1.3.1/lib/active_record/tasks/database_tasks.rb:368:in `reconstruct_from_schema'
/var/lib/gems/2.5.0/gems/activerecord-6.1.3.1/lib/active_record/test_databases.rb:17:in `block in create_and_load_schema'
/var/lib/gems/2.5.0/gems/activerecord-6.1.3.1/lib/active_record/test_databases.rb:14:in `each'
/var/lib/gems/2.5.0/gems/activerecord-6.1.3.1/lib/active_record/test_databases.rb:14:in `create_and_load_schema'
I'll take a look at these methods soon and see if there's a way to ignore the postgis specific tables from this.
@Lapizistik I made this modification (https://github.com/rgeo/activerecord-postgis-adapter/commit/074506fd1a4d19b455628845bc573c8d1ba59644) I'm pretty sure it will work across all 6.x versions, but I only tested this off the master branch so it's 6.1.x compatible. If you could give it a try that would be great.
@keithdoggett I tried your modification gem 'activerecord-postgis-adapter', git: 'https://github.com/rgeo/activerecord-postgis-adapter.git', branch: 'ignore-srs-truncate'
, the tests work fine.
I don't know whether there are any side-effects I am not aware of.
Thaks a lot again!
Klaus
Fixed by #341
Description
I try to use this gem on a fresh rails 6 project with an ordinary postgres database user (that has no superuser rights). I therefore add the postgis extension to the database via postgres template (see below). This works for development mode, but running
rails test
fails (sometimes the first run succeeds) withDRb::DRbRemoteError: PG::InsufficientPrivilege: ERROR: permission denied for table spatial_ref_sys
(sometimes also for tablear_internal_metadata
instead). Stopping or killingspring
does not help.The setup described below fails with rails 6.1.3 and 6.0.3.1. The identical setup works with rails 5.2.5 without errors. It also works for rails 6 if I use a superuser role.
This is my first time using activerecord-postgis-adapter and I am not an expert in how rails 6 runs tests (as far as I could see it creates at least 4 test databases using them in parallel?) so maybe I am doing something wrong?
Thanks for your work!
Setup
ruby version: 2.5 OS: debian buster postgresql: 11.11-0+deb10u1 postgis: 2.5.1+dfsg-1
postgres (psql)
First create a postgis template in postgres:
And some user:
rails (6.1.3)
Following the instructions in the README:
config/database.yml
Logfiles
The
log/test.log
shows lots ofAnd
/var/log/postgresql/postgresql-11-main.log
gives