DMPRoadmap / roadmap

DCC/UC3 collaboration for a data management planning tool
MIT License
102 stars 110 forks source link

Template Migration fail #2070

Closed Ebrahim1010 closed 5 years ago

Ebrahim1010 commented 5 years ago

Please complete the following fields as applicable:

Expected behaviour:

Actual behaviour: Formatting is messed, migration fails due to invalid characters

(1,'What data will you collect or create?',NULL,1,1,'2019-03-26 11:58:46','2019-03-26 11:58:46',1,1,0,'48f21e93-6016-4601-8d0a-4966d79e9b1b'),

(2,'How will the data be collected or created?',NULL,2,1,'2019-03-26 11:58:46','2019-03-26 11:58:46',1,1,0,'8b399c76-0fac-41dd-88ba-f98d239ae6cc'),

(85,'Expected dataset types:\r\n\r\n\r\nWhat kind of data will you create? List briefly with a short description where necessary.\r\n','','',1,NULL,NULL,NULL,68,'2015-11-16 10:09:41','2015-11-23 12:50:38',1),

(86,'Data Volume\r\nExpect to generate more than 1TB of data: Y/N (tick box)\r\n','','',2,NULL,NULL,NULL,68,'2015-11-16 10:12:24','2015-11-23 12:50:57',4),

-existing database Steps to reproduce:

xsrust commented 5 years ago

Hi,

Which part of which migration is failing? Are you still working on MigrationStep1 or have you moved past that? Would it be possible to list the error message?

Ebrahim1010 commented 5 years ago

Hi,

I'm going to start a new migration shortly. I will post the error i'm getting shortly.

Ebrahim1010 commented 5 years ago

db migrate: Upgrade step 2 error:

beginning number 45 rake aborted! StandardError: An error has occurred, all later migrations canceled:

Mysql2::Error: Incorrect string value: '\xE2\x97\x8F Do...' for column 'guidance' at row 1: INSERT INTO new_questions (text, default_value, guidance, number, new_section_id, question_format_id, modifiable, question_id, updated_at, created_at) VALUES ('4.2 Data Volume\r\nDo you require data storage?', '', '

Questions to consider:
● Do you have enough storage including space required to accommodate backups?
● Will the size of the datasets impact/limit the ways in which they can be shared?

\r\n

All storage space must be budgeted for and included in your funding proposal. For more information about storage contact ICTS.
(Include option to share note with collaborators)

', 2, 207, 3, 1, 342, '2016-01-15 13:35:01', '2016-01-15 13:35:01') /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in _query' /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:inblock in query' /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in handle_interrupt' /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:inquery' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in block in execute' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:484:inblock in log' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in instrument' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:478:inlog' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in execute' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:231:inexecute' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:248:in exec_insert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:108:ininsert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/query_cache.rb:14:in insert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation.rb:64:ininsert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/persistence.rb:524:in _create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/counter_cache.rb:139:in_create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/locking/optimistic.rb:75:in _create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/attribute_methods/dirty.rb:133:in_create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:306:in block in _create_record' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:117:incall' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:555:in block (2 levels) in compile' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:505:incall' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:92:in __run_callbacks__' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:778:in_run_create_callbacks' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:306:in _create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/timestamp.rb:57:in_create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/persistence.rb:504:in create_or_update' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:302:inblock in create_or_update' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:117:in call' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:555:inblock (2 levels) in compile' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:505:in call' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:92:inrun_callbacks__' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:778:in _run_save_callbacks' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:302:increate_or_update' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/persistence.rb:142:in save!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/validations.rb:43:insave!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/attribute_methods/dirty.rb:29:in save!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:291:inblock in save!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:351:in block in with_transaction_returning_status' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:211:intransaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:220:in transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:348:inwith_transaction_returning_status' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:291:in save!' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:288:inblock (5 levels) in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in each' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:ineach' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:286:in block (4 levels) in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:ineach' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in each' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:278:inblock (3 levels) in up' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:270:in each' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:270:inblock (2 levels) in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:51:in block (2 levels) in find_each' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:51:ineach' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:51:in block in find_each' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:124:infind_in_batches' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:50:in find_each' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:220:inblock in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in block in transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/transaction.rb:184:inwithin_new_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:220:intransaction' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:219:in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:611:inexec_migration' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in block (2 levels) in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:inblock in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in with_connection' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:inmigrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:inblock in execute_migration_in_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in ddl_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:inexecute_migration_in_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in block in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:ineach' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:inup' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:inmigrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in block (2 levels) in <top (required)>' /var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in<top (required)>' ActiveRecord::StatementInvalid: Mysql2::Error: Incorrect string value: '\xE2\x97\x8F Do...' for column 'guidance' at row 1: INSERT INTO new_questions (text, default_value, guidance, number, new_section_id, question_format_id, modifiable, question_id, updated_at, created_at) VALUES ('4.2 Data Volume\r\nDo you require data storage?', '', '

Questions to consider:
● Do you have enough storage including space required to accommodate backups?
● Will the size of the datasets impact/limit the ways in which they can be shared?

\r\n

All storage space must be budgeted for and included in your funding proposal. For more information about storage contact ICTS.
(Include option to share note with collaborators)

', 2, 207, 3, 1, 342, '2016-01-15 13:35:01', '2016-01-15 13:35:01') /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in _query' /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:inblock in query' /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in handle_interrupt' /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:inquery' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in block in execute' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:484:inblock in log' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in instrument' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:478:inlog' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in execute' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:231:inexecute' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:248:in exec_insert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:108:ininsert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/query_cache.rb:14:in insert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation.rb:64:ininsert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/persistence.rb:524:in _create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/counter_cache.rb:139:in_create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/locking/optimistic.rb:75:in _create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/attribute_methods/dirty.rb:133:in_create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:306:in block in _create_record' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:117:incall' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:555:in block (2 levels) in compile' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:505:incall' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:92:in `
run_callbacks' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:778:in _run_create_callbacks' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:306:in_create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/timestamp.rb:57:in _create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/persistence.rb:504:increate_or_update' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:302:in block in create_or_update' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:117:incall' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:555:in block (2 levels) in compile' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:505:incall' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:92:in `run_callbacks' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:778:in _run_save_callbacks' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:302:increate_or_update' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/persistence.rb:142:in save!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/validations.rb:43:insave!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/attribute_methods/dirty.rb:29:in save!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:291:inblock in save!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:351:in block in with_transaction_returning_status' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:211:intransaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:220:in transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:348:inwith_transaction_returning_status' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:291:in save!' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:288:inblock (5 levels) in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in each' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:ineach' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:286:in block (4 levels) in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:ineach' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in each' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:278:inblock (3 levels) in up' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:270:in each' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:270:inblock (2 levels) in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:51:in block (2 levels) in find_each' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:51:ineach' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:51:in block in find_each' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:124:infind_in_batches' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:50:in find_each' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:220:inblock in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in block in transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/transaction.rb:184:inwithin_new_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:220:intransaction' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:219:in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:611:inexec_migration' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in block (2 levels) in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:inblock in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in with_connection' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:inmigrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:inblock in execute_migration_in_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in ddl_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:inexecute_migration_in_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in block in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:ineach' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:inup' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:inmigrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in block (2 levels) in <top (required)>' /var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in<top (required)>' Mysql2::Error: Incorrect string value: '\xE2\x97\x8F Do...' for column 'guidance' at row 1 /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in _query' /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:inblock in query' /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in handle_interrupt' /var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:inquery' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in block in execute' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:484:inblock in log' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in instrument' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:478:inlog' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in execute' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:231:inexecute' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:248:in exec_insert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:108:ininsert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/query_cache.rb:14:in insert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation.rb:64:ininsert' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/persistence.rb:524:in _create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/counter_cache.rb:139:in_create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/locking/optimistic.rb:75:in _create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/attribute_methods/dirty.rb:133:in_create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:306:in block in _create_record' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:117:incall' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:555:in block (2 levels) in compile' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:505:incall' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:92:in `run_callbacks' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:778:in _run_create_callbacks' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:306:in_create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/timestamp.rb:57:in _create_record' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/persistence.rb:504:increate_or_update' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:302:in block in create_or_update' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:117:incall' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:555:in block (2 levels) in compile' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:505:incall' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:92:in `run_callbacks__' /var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/callbacks.rb:778:in _run_save_callbacks' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/callbacks.rb:302:increate_or_update' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/persistence.rb:142:in save!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/validations.rb:43:insave!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/attribute_methods/dirty.rb:29:in save!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:291:inblock in save!' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:351:in block in with_transaction_returning_status' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:211:intransaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:220:in transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:348:inwith_transaction_returning_status' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:291:in save!' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:288:inblock (5 levels) in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in each' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:ineach' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:286:in block (4 levels) in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:ineach' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in each' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:278:inblock (3 levels) in up' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:270:in each' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:270:inblock (2 levels) in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:51:in block (2 levels) in find_each' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:51:ineach' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:51:in block in find_each' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:124:infind_in_batches' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/batches.rb:50:in find_each' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:220:inblock in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in block in transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/transaction.rb:184:inwithin_new_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:220:intransaction' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:219:in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:611:inexec_migration' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in block (2 levels) in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:inblock in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in with_connection' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:inmigrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:inblock in execute_migration_in_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in ddl_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:inexecute_migration_in_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in block in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:ineach' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:inup' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:inmigrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in block (2 levels) in <top (required)>' /var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in<top (required)>' Tasks: TOP => db:migrate (See full trace by running task with --trace)

xsrust commented 5 years ago

Hi,

It looks like the issue here is with Mysql, and given that the error is with 'Incorrect String Value', I'm guessing that this is an issue with the charactor set. We use Postgres at the DCC (so havent run into this issue) but @briri runs a Mysql backed application and I remember he had to change to use utf8-mb4 as their charactor set as it encodes properly with 4 chars instead of 3.

Alternatively, those bullet points are causing the issue, so they could be stripped out.

Ebrahim1010 commented 5 years ago

Hi,

okay thanks, let me give it a shot

briri commented 5 years ago

Hey @xsrust and @Ebrahim1010 We're using Encoding: UTF8 - Unicode (utf8) and Collation: utf8_unicode_ci here at the database level. This is due to the implications of converting everything from 3 to 4 bytes and MySQL's limitations on the maximum size of indexes which is 768 bytes. Several of the indexes DMPRoadmap uses would break if you set the entire DB to utfmb4.

I recommend updating the encoding/collation on a few particular columns/tables instead. For example:

ALTER TABLE `new_questions` MODIFY `text` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE `questions` MODIFY `text` TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

You may want to also do the same for the text column on the answers and guidances tables and possibly the description fields on guidance_groups, themes, plans, templates, sections as well.

Ebrahim1010 commented 5 years ago

Thank you, I'm busy working on that.

I have a new error that I need help with please.

New error

Mysql2::Error: Table 'templates' already exists: CREATE TABLE templates (id int(11) auto_increment PRIMARY KEY, title varchar(255), description text, published tinyint(1), organisation_id int(11), locale varchar(255), is_default tinyint(1), created_at datetime, updated_at datetime, version int(11), visibility int(11), customization_of int(11), dmptemplate_id int(11), migrated tinyint(1)) ENGINE=InnoDB

Ebrahim1010 commented 5 years ago

Okay error above resolved: added the following to the top of the migration file.

class NewPlanTemplateStructure < ActiveRecord::Migration def up

new template tables

drop_table :templates
drop_table :new_phases
drop_table :new_sections
drop_table :new_questions
drop_join_table :new_questions, :themes
drop_table :new_answers
drop_table :question_options
drop_join_table :new_answers, :question_options
drop_table :notes
drop_table :annotations
drop_table :new_plans
drop_table :roles
drop_table :new_plans_guidance_groups
Ebrahim1010 commented 5 years ago

New error: -- create_table(:templates) W, [2019-03-28T11:59:39.092653 #23058] WARN -- : DEPRECATION WARNING: #timestamps was called without specifying an option for null. In Rails 5, this behavior will change to null: false. You should manually specify null: true to prevent the behavior of your existing migrations from changing. (called from block in up at /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:25) -> 0.0103s -- create_table(:new_phases) W, [2019-03-28T11:59:39.102358 #23058] WARN -- : DEPRECATION WARNING: #timestamps was called without specifying an option for null. In Rails 5, this behavior will change to null: false. You should manually specify null: true to prevent the behavior of your existing migrations from changing. (called from block in up at /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:39) -> 0.0084s -- create_table(:new_sections) W, [2019-03-28T11:59:39.110881 #23058] WARN -- : DEPRECATION WARNING: #timestamps was called without specifying an option for null. In Rails 5, this behavior will change to null: false. You should manually specify null: true to prevent the behavior of your existing migrations from changing. (called from block in up at /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:50) -> 0.0410s -- create_table(:new_questions) W, [2019-03-28T11:59:39.152221 #23058] WARN -- : DEPRECATION WARNING: #timestamps was called without specifying an option for null. In Rails 5, this behavior will change to null: false. You should manually specify null: true to prevent the behavior of your existing migrations from changing. (called from block in up at /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:63) -> 0.0083s -- create_join_table(:new_questions, :themes) -> 0.0349s -- create_table(:new_answers) W, [2019-03-28T11:59:39.196068 #23058] WARN -- : DEPRECATION WARNING: #timestamps was called without specifying an option for null. In Rails 5, this behavior will change to null: false. You should manually specify null: true to prevent the behavior of your existing migrations from changing. (called from block in up at /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:79) -> 0.0095s -- create_table(:question_options) W, [2019-03-28T11:59:39.204904 #23058] WARN -- : DEPRECATION WARNING: #timestamps was called without specifying an option for null. In Rails 5, this behavior will change to null: false. You should manually specify null: true to prevent the behavior of your existing migrations from changing. (called from block in up at /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:88) -> 0.0234s -- create_join_table(:new_answers, :question_options) -> 0.0233s -- create_table(:notes) W, [2019-03-28T11:59:39.252421 #23058] WARN -- : DEPRECATION WARNING: #timestamps was called without specifying an option for null. In Rails 5, this behavior will change to null: false. You should manually specify null: true to prevent the behavior of your existing migrations from changing. (called from block in up at /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:100) -> 0.0086s -- create_table(:annotations) W, [2019-03-28T11:59:39.260521 #23058] WARN -- : DEPRECATION WARNING: #timestamps was called without specifying an option for null. In Rails 5, this behavior will change to null: false. You should manually specify null: true to prevent the behavior of your existing migrations from changing. (called from block in up at /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:108) -> 0.0106s -- create_table(:new_plans) W, [2019-03-28T11:59:39.271284 #23058] WARN -- : DEPRECATION WARNING: #timestamps was called without specifying an option for null. In Rails 5, this behavior will change to null: false. You should manually specify null: true to prevent the behavior of your existing migrations from changing. (called from block in up at /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:116) -> 0.0492s -- create_table(:new_plans_guidance_groups) -> 0.0085s -- create_table(:roles) W, [2019-03-28T11:59:39.329257 #23058] WARN -- : DEPRECATION WARNING: #timestamps was called without specifying an option for null. In Rails 5, this behavior will change to null: false. You should manually specify null: true to prevent the behavior of your existing migrations from changing. (called from block in up at /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:139) -> 0.0226s -- change_table(:projects) rake aborted! StandardError: An error has occurred, all later migrations canceled:

Index name 'index_projects_on_dmptemplate_id' on table 'projects' already exists /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:954:in add_index_options' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:540:inadd_index' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_definitions.rb:461:in index' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:143:inblock in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:377:in change_table' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:665:inblock in method_missing' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in block in say_with_time' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:insay_with_time' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:654:in method_missing' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:142:inup' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:611:in exec_migration' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:inblock (2 levels) in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:in block in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:inwith_connection' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:inmigrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:in block in execute_migration_in_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:inddl_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:in execute_migration_in_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:inblock in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in each' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:inmigrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:inmigrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:inblock (2 levels) in <top (required)>' /var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in <top (required)>' ArgumentError: Index name 'index_projects_on_dmptemplate_id' on table 'projects' already exists /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:954:inadd_index_options' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:540:in add_index' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_definitions.rb:461:inindex' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:143:in block in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:377:inchange_table' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:665:in block in method_missing' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:inblock in say_with_time' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in say_with_time' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:654:inmethod_missing' /opt/roadmap/db/migrate/20161122152339_new_plan_template_structure.rb:142:in up' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:611:inexec_migration' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in block (2 levels) in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:inblock in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in with_connection' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:inmigrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:inblock in execute_migration_in_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in ddl_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:inexecute_migration_in_transaction' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in block in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:ineach' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:inup' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in migrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:inmigrate' /var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in block (2 levels) in <top (required)>' /var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in<top (required)>' Tasks: TOP => db:migrate

xsrust commented 5 years ago

Ok, That above error would be caused by those intermediary tables sticking around after a failed migration. You'll want to remove those lines if you re-start from scratch again.

The error you have now is because those indexes were already applied to the other table as well

Ebrahim1010 commented 5 years ago

Okay thank you, I removed the indexes and re ran the migration. Templates have been migrated successfully.

Ebrahim1010 commented 5 years ago

Upgrade step 3: new error

-- add_foreign_key(:users_perms, :users)

D, [2019-03-29T12:18:33.796143 #29380] DEBUG -- :    (1.1ms)  ALTER TABLE users_perms ADD CONSTRAINT fk_rails_b46072b3a6

FOREIGN KEY (user_id)

  REFERENCES users (id)

 

rake aborted!

StandardError: An error has occurred, all later migrations canceled:

 

Mysql2::Error: Table 'roadmap_development.users_perms' doesn't exist: ALTER TABLE users_perms ADD CONSTRAINT fk_rails_b46072b3a6

FOREIGN KEY (user_id)

  REFERENCES users (id)

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `_query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `block in query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `handle_interrupt'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `query'

/var/lib/gems/2.3.0/gems/rack-mini-profiler-0.10.1/lib/patches/db/mysql2.rb:20:in `query'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `block in execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'

/var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:231:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:776:in `add_foreign_key'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:665:in `block in method_missing'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `block in say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:654:in `method_missing'

/opt/roadmap/db/migrate/20161206122926_add_foreign_keys.rb:79:in `change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:608:in `exec_migration'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in `ddl_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:in `up'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'

ActiveRecord::StatementInvalid: Mysql2::Error: Table 'roadmap_development.users_perms' doesn't exist: ALTER TABLE users_perms ADD CONSTRAINT fk_rails_b46072b3a6

FOREIGN KEY (user_id)

  REFERENCES users (id)

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `_query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `block in query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `handle_interrupt'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `query'

/var/lib/gems/2.3.0/gems/rack-mini-profiler-0.10.1/lib/patches/db/mysql2.rb:20:in `query'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `block in execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'

/var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:231:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:776:in `add_foreign_key'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:665:in `block in method_missing'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `block in say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:654:in `method_missing'

/opt/roadmap/db/migrate/20161206122926_add_foreign_keys.rb:79:in `change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:608:in `exec_migration'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in `ddl_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:in `up'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'

Mysql2::Error: Table 'roadmap_development.users_perms' doesn't exist

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `_query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `block in query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `handle_interrupt'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `query'

/var/lib/gems/2.3.0/gems/rack-mini-profiler-0.10.1/lib/patches/db/mysql2.rb:20:in `query'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `block in execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'

/var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:231:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:776:in `add_foreign_key'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:665:in `block in method_missing'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `block in say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:654:in `method_missing'

/opt/roadmap/db/migrate/20161206122926_add_foreign_keys.rb:79:in `change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:608:in `exec_migration'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in `ddl_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:in `up'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'

Tasks: TOP => db:migrate

(See full trace by running task with --trace)

01465524@devubudmp002:/opt/roadmap$

I hope that rerunning this migration isn't going to complain about the stuff it added and didn't remove because the migration failed

xsrust commented 5 years ago

hi @Ebrahim1010, I'm glad to hear you've made it to step3. It looks like your missing the users_perms table which should have been added by:

https://github.com/DMPRoadmap/roadmap/blob/dmponline4_upgrade_step3/db/migrate/20161115143222_rename_roles_to_perms.rb

Did you add in code to explicitly remove this table?

Also, re-running the migration will definitely complain about you removing the intermediary tables if they do not exist from previous failures. You'll have to either re-tailor the migration each time in rails, or, my perfered approach when developing this was to take a db dump of my data between each step, that way I could re-load the last known-good, and attempt to update my data to conform to the migrations.

Ebrahim1010 commented 5 years ago

@xsrust Yes thank you. I'm moving along well with your help. I was just wondering why you don't write migration files with validation or roll back if unsuccessful.

I have another error that I need some help with though: translation missing: en-UK.activerecord.errors.messages.record_invalid

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/validations.rb:79:in `raise_record_invalid'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/validations.rb:43:in `save!'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/attribute_methods/dirty.rb:29:in `save!'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:291:in `block in save!'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:351:in `block in with_transaction_returning_status'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `block in transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/transaction.rb:184:in `within_new_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:220:in `transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:348:in `with_transaction_returning_status'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:291:in `save!'

/opt/roadmap/db/migrate/20170130173049_add_option_based_to_question_formats.rb:10:in `block in change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in `each'

/opt/roadmap/db/migrate/20170130173049_add_option_based_to_question_formats.rb:7:in `change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:608:in `exec_migration'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in `ddl_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:in `up'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'

ActiveRecord::RecordInvalid: translation missing: en-UK.activerecord.errors.messages.record_invalid

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/validations.rb:79:in `raise_record_invalid'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/validations.rb:43:in `save!'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/attribute_methods/dirty.rb:29:in `save!'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:291:in `block in save!'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:351:in `block in with_transaction_returning_status'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `block in transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/transaction.rb:184:in `within_new_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:220:in `transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:348:in `with_transaction_returning_status'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/transactions.rb:291:in `save!'

/opt/roadmap/db/migrate/20170130173049_add_option_based_to_question_formats.rb:10:in `block in change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in `each'

/opt/roadmap/db/migrate/20170130173049_add_option_based_to_question_formats.rb:7:in `change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:608:in `exec_migration'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in `ddl_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:in `up'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'

Tasks: TOP => db:migrate

Ebrahim1010 commented 5 years ago

Hi guys, Any assistance with the above would be appreciated. the upgrade is almost done.

Thank you

briri commented 5 years ago

Hey @Ebrahim1010,

Based on the error above it looks like you may have some bad data in your question_formats table. Make sure that the title, format_type, created_at and updated_at columns have values for each record. Also make sure that the format_type is unique for each one.

Ebrahim1010 commented 5 years ago

Hey @briri Thank you! I made some clean ups and it worked. Although I didn't have a format_type column, I had a description column though. There were duplicates fields which I deleted and it worked. I'm onto step 4 now!

Ebrahim1010 commented 5 years ago

I have another error: D, [2019-04-03T10:34:50.085961 #23949] DEBUG -- :    (0.2ms)  SHOW CREATE TABLE users_perms

rake aborted!

StandardError: An error has occurred, all later migrations canceled:

 

Table 'users_perms' has no foreign key on column 'perm_id'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:808:in `block in remove_foreign_key'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:802:in `fetch'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:802:in `remove_foreign_key'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:665:in `block in method_missing'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `block in say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:654:in `method_missing'

/opt/roadmap/db/migrate/20170702012742_ensure_indexes_in_place.rb:4:in `change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:608:in `exec_migration'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in `ddl_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:in `up'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'

ArgumentError: Table 'users_perms' has no foreign key on column 'perm_id'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:808:in `block in remove_foreign_key'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:802:in `fetch'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/schema_statements.rb:802:in `remove_foreign_key'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:665:in `block in method_missing'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `block in say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:654:in `method_missing'

/opt/roadmap/db/migrate/20170702012742_ensure_indexes_in_place.rb:4:in `change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:608:in `exec_migration'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in `ddl_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:in `up'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'

Tasks: TOP => db:migrate

xsrust commented 5 years ago

Hi,

The error says that migration expects a foreign key on column perm_id, but you dont have one there somehow. Maybe check your db against the assumptions made in the migration before running, and comment out lines which will not behave as expected.

That foreign key should have been added in an earlier migration: https://github.com/DMPRoadmap/roadmap/blob/master/db/migrate/20161206122926_add_foreign_keys.rb#L79 So I'm not sure how you got into that state.

I hope that helps.

Ebrahim1010 commented 5 years ago

Okay I manually created one manually and that worked. I'm moving on to upgrading to v1.0.0!

Received this error now: == 20180123161959 ChangeLongStringsToText: migrating ==========================

-- change_column(:orgs, :links, :text)

rake aborted!

StandardError: An error has occurred, all later migrations canceled:

 

Mysql2::Error: BLOB, TEXT, GEOMETRY or JSON column 'links' can't have a default value: ALTER TABLE orgs CHANGE links links text DEFAULT '[]'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `_query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `block in query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `handle_interrupt'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `query'

/var/lib/gems/2.3.0/gems/rack-mini-profiler-0.10.1/lib/patches/db/mysql2.rb:20:in `query'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `block in execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'

/var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:231:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:531:in `change_column'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:665:in `block in method_missing'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `block in say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:654:in `method_missing'

/opt/roadmap/db/migrate/20180123161959_change_long_strings_to_text.rb:3:in `change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:608:in `exec_migration'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in `ddl_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:in `up'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'

ActiveRecord::StatementInvalid: Mysql2::Error: BLOB, TEXT, GEOMETRY or JSON column 'links' can't have a default value: ALTER TABLE orgs CHANGE links links text DEFAULT '[]'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `_query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `block in query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `handle_interrupt'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `query'

/var/lib/gems/2.3.0/gems/rack-mini-profiler-0.10.1/lib/patches/db/mysql2.rb:20:in `query'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `block in execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'

/var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:231:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:531:in `change_column'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:665:in `block in method_missing'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `block in say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:654:in `method_missing'

/opt/roadmap/db/migrate/20180123161959_change_long_strings_to_text.rb:3:in `change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:608:in `exec_migration'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in `ddl_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:in `up'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'

Mysql2::Error: BLOB, TEXT, GEOMETRY or JSON column 'links' can't have a default value

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `_query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:80:in `block in query'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `handle_interrupt'

/var/lib/gems/2.3.0/gems/mysql2-0.3.21/lib/mysql2/client.rb:79:in `query'

/var/lib/gems/2.3.0/gems/rack-mini-profiler-0.10.1/lib/patches/db/mysql2.rb:20:in `query'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `block in execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'

/var/lib/gems/2.3.0/gems/activesupport-4.2.7/lib/active_support/notifications/instrumenter.rb:20:in `instrument'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/mysql2_adapter.rb:231:in `execute'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:531:in `change_column'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:665:in `block in method_missing'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `block in say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:634:in `say_with_time'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:654:in `method_missing'

/opt/roadmap/db/migrate/20180123161959_change_long_strings_to_text.rb:3:in `change'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:608:in `exec_migration'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:592:in `block (2 levels) in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:591:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:292:in `with_connection'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:590:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:768:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:998:in `block in execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:1046:in `ddl_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:997:in `execute_migration_in_transaction'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:959:in `block in migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:955:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:823:in `up'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/migration.rb:801:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/tasks/database_tasks.rb:137:in `migrate'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/railties/databases.rake:44:in `block (2 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'

Tasks: TOP => db:migrate

xsrust commented 5 years ago

Ahh, that's another one Briri struggled with. We use Postgres at the DCC, but CDL uses Mysql, which dosent support defaults for some fields.

We've since commented out that migration, but had already run it so didnt want to remove it from the project. https://github.com/DMPRoadmap/roadmap/blob/master/db/migrate/20180123161959_change_long_strings_to_text.rb

Simply comment out the body of that migration and you should be good.

There's a later one to address this by removing the defaults as well, but this should not cause issues if you comment the one which is producing the error: https://github.com/DMPRoadmap/roadmap/blob/master/db/migrate/20180713145547_remove_defaults_from_links.rb

Ebrahim1010 commented 5 years ago

@xsrust Thank you! that worked

Ebrahim1010 commented 5 years ago

Do you have a fix for this error:

01465524@devubudmp002:/opt/roadmap$ sudo bundle exec rake upgrade:v1_0_0

W, [2019-04-03T13:00:41.345015 #24893]  WARN -- : You are setting a key that conflicts with a built-in method OmniAuth::AuthHash::InfoHash#name defined at /var/lib/gems/2.3.0/gems/omniauth-1.3.2/lib/omniauth/auth_hash.rb:34. This can cause unexpected behavior when accessing the key as a property. You can still access the key via the #[] method.

rake aborted!

NoMethodError: undefined method `option_based?' for nil:NilClass

/opt/roadmap/app/models/answer.rb:77:in `is_valid?'

/opt/roadmap/app/models/plan.rb:794:in `block in no_questions_matches_no_answers?'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:132:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:132:in `reduce'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:132:in `public_send'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:132:in `method_missing'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:99:in `method_missing'

/opt/roadmap/app/models/plan.rb:793:in `no_questions_matches_no_answers?'

/opt/roadmap/lib/tasks/upgrade.rake:79:in `block (3 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in `each'

/var/lib/gems/2.3.0/gems/activerecord-4.2.7/lib/active_record/relation/delegation.rb:46:in `each'

/opt/roadmap/lib/tasks/upgrade.rake:78:in `block (2 levels) in <top (required)>'

/opt/roadmap/lib/tasks/upgrade.rake:8:in `block (2 levels) in <top (required)>'

/var/lib/gems/2.3.0/gems/rake-11.3.0/exe/rake:27:in `<top (required)>'

Tasks: TOP => upgrade:v1_0_0

(See full trace by running task with --trace)

xsrust commented 5 years ago

Ok, starting at the bottom of the call chain, your error is being thrown on line 79 of upgrade.rake: https://github.com/DMPRoadmap/roadmap/blob/7814e538ec8fb3542ef4e37968153d698ab0d005/lib/tasks/upgrade.rake#L79

This is calling the function no_questions_matches_no_answers on the plan model: https://github.com/DMPRoadmap/roadmap/blob/7814e538ec8fb3542ef4e37968153d698ab0d005/app/models/plan.rb#L788-L799

Next, you can see that it's throwing an error on the is_valid? function for answer: https://github.com/DMPRoadmap/roadmap/blob/7814e538ec8fb3542ef4e37968153d698ab0d005/app/models/answer.rb#L75-L84

Finally, the error actually thrown is NoMethodError: undefined methodoption_based?' for nil:NilClass This implies that the call tooption_based?` is being issued to an instance of nil. Given the code where this is triggered, it looks like you have questions in your database which do not have questionformats, but the code expects this data to be there.

My approach would be to fixup your data here rather than modifying the migration.

norman-smith commented 5 years ago

Hi, Ebrahim1010 and I have been working on this migration and I would like to know the extent of the data fixup? Could you provide an example of the question formats you stated above. it seems the migration fails on setting each plan to complete provided it passes some validation check. I could attempt to fix the data directly in the database, but I need to know what the migration job wants the data to look like before I can do it.

xsrust commented 5 years ago

Good morning,

Ok, the error above has more to do with existance than validity. It's caused by the questionformat being nil, so none is set for that question. Essentially, all questions should have a question_format selected.

rails c from the command line inside your project directory will launch a rails console to enable you to interact with your data. Question.includes(:question_format).where(question_format: {id: nil}) will select for you all questions which do not currently have a question format set.

Let me know if that clears things up for you.

norman-smith commented 5 years ago

Hi Sam

Thanks so much for this. It has returned 5 questions. Comparing it to the other questions and the questions_formats table the entry for those 5 questions are actually invalid. It’s using a question_format of 7, but the question_formats table ranges from 0 to 6. How would I determine which question format to set it to without visually seeing the question?

Regards Norman

From: Sam Rust notifications@github.com Sent: Thursday, 04 April 2019 10:49 To: DMPRoadmap/roadmap roadmap@noreply.github.com Cc: Norman Smith norman.smith@uct.ac.za; Comment comment@noreply.github.com Subject: Re: [DMPRoadmap/roadmap] Template Migration fail (#2070)

Good morning,

Ok, the error above has more to do with existance than validity. It's caused by the questionformat being nil, so none is set for that question. Essentially, all questions should have a question_format selected.

rails c from the command line inside your project directory will launch a rails console to enable you to interact with your data. Question.includes(:question_format).where(question_format: {id: nil}) will select for you all questions which do not currently have a question set.

Let me know if that clears things up for you.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://protect-za.mimecast.com/s/C7gnCwjyBrSX84Z8fVXz-J, or mute the threadhttps://protect-za.mimecast.com/s/alB3CxGzD1UKMgqMFvsB0k.

Disclaimer - University of Cape Town This email is subject to UCT policies and email disclaimer published on our website at http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650 9111. If this email is not related to the business of UCT, it is sent by the sender in an individual capacity. Please report security incidents or abuse via https://csirt.uct.ac.za/page/report-an-incident.php.

xsrust commented 5 years ago

Hi Norman,

You can certainly output the question's question.text for each of these to view the question's content. Alternatively, just default them all to be a 'textarea' with:

textarea = QuestionFormat.find_by(formattype: QuestionFormat.formattypes[:textarea])
Question.includes(:question_format).where(question_format: {id: nil}).each do |question|
    question.question_format = textarea
    question.save
end
norman-smith commented 5 years ago

Worked like a bomb. Thanks so much.

norman-smith commented 5 years ago

What are your recommendations for testing version 1 at present. Starting the server and browsing to the site produces the following error.

Processing by HomeController#index as HTML Rendered home/_welcome.html.erb (4.7ms) Rendered home/index.html.erb within layouts/application (9.7ms) Completed 500 Internal Server Error in 33ms (ActiveRecord: 0.0ms)

ActionView::Template::Error (undefined method each' for nil:NilClass): 4: <%= raw _('<p>%{application_name} has been developed by the <strong>%{organisation_name}</strong> to help you write data management plans.</p>') % {:application_name => Rails.configuration.branding[:application][:name], :organisation_name => Rails.configuration.branding[:organisation][:name]} %> 5: <label for="linklist"><%=_('Getting started:')%></label> 6: <ul id="linklist"> 7: <% Rails.application.config.branding[:organisation][:welcome_links].each do |wlink| %> 8: <li><a href="<%= wlink[:url] %>"><%= wlink[:title] %></a></li> 9: <% end %> 10: </ul> app/views/home/_welcome.html.erb:7:in_app_views_homewelcome_html_erb1452276979964544765_45979500' app/views/home/index.html.erb:3:in `_app_views_home_index_html_erb___286543724646346025_46360280'

Rendered /var/lib/gems/2.3.0/gems/actionpack-4.2.7/lib/action_dispatch/middleware/templates/rescues/_source.erb (7.6ms) Rendered /var/lib/gems/2.3.0/gems/actionpack-4.2.7/lib/action_dispatch/middleware/templates/rescues/_trace.html.erb (9.5ms) Rendered /var/lib/gems/2.3.0/gems/actionpack-4.2.7/lib/action_dispatch/middleware/templates/rescues/_request_and_response.html.erb (0.9ms) Rendered /var/lib/gems/2.3.0/gems/actionpack-4.2.7/lib/action_dispatch/middleware/templates/rescues/template_error.html.erb within rescues/layout (29.5ms)

xsrust commented 5 years ago

It looks like the content of your config/branding.yml is different than that expected by the project. There should be an example file from the project there that you'll need to update (and remove example from the name of) to reflect your configuration.

norman-smith commented 5 years ago

Nice one, getting somewhere now. I think I need to recompile assets before it'll look right. Thanks for this.

Ebrahim1010 commented 5 years ago

Hi Sam,

We have completed the migration to roadmap v.1. Thanks for your help! Can we move straight from v1 to v2 using the wiki or is there another approach? Thank you

briri commented 5 years ago

Hi @Ebrahim1010 and @norman-smith,

You should run through each of the version releases from v1.0.1 to the latest. Be sure to read the release notes for each. Many of them require to run additional rake tasks to introduce new tables/columns or fix up bad data. The upgrade to v2 in particular is quite involved but has more scripts in place to make the process easier.

Ebrahim1010 commented 5 years ago

@briri okay thanks.

So Below assets we have an archive. Do we need to download the archive and merge it with our current dmproadmap folder structure?

xsrust commented 5 years ago

Those archives are just for convienence if you were otherwise deploying the release. When upgrading through, git checkout <tag> while on the master branch will get you up to date with that release, and you'll be able to follow the instructions for that release.

Ebrahim1010 commented 5 years ago

Hi @xsrust, @briri

We are currently transitioning from v1.1.4 to v2.0.0 and have some cleanups to do. The following are some invalid records that we need some help with. How do we identify where these records are?

Checking Exported Plan records 545 orphaned records due to nil or missing plan

Checking Language records 5 records that do not have valid [:abbreviation]

Thanks

xsrust commented 5 years ago

Hi @Ebrahim1010

For the former, It says that it's checking the ExportedPlan records. Of those, 545 either have a plan_id of nil, OR the plan which they reference does not exist. You can either find these a bit more readably by iterating over all ExportedPlan objects and checking that each has a valid plan, ex:

::ExportedPlan.all.each do |ep|
  unless ep.plan.present?
    ep.destroy!
  end
end

OR, you can do it a bit more performantly by left outer joining these on the db, which will cause any broken associations to be nil in the result. You can then query that the plan.id field is nil, and destroy these.

::ExportedPlan
            .joins("LEFT OUTER JOIN plans on plans.id = exported_plans.plan_id")
.where(plans: { id: nil }).each do |ep|
    ep.destroy!
end

For the languages, check the model for the definition of a vailid abbreviation: https://github.com/DMPRoadmap/roadmap/blob/c7291b470d19bb29f59c79c45f81e0210a40e7c6/app/models/language.rb#L46-L49 Then you'll need to see in what way yours do not conform. I believe there was a rake task added as well to normalize these, rake upgrade:normalize_language_formats.

Ebrahim1010 commented 5 years ago

Hi Sam,

Thank you for the help!

I wanted some clarification on the exported plans and destroying them. By destroying the exported plans does it mean that we are deleting the plans in the databse that have been created and exported as opposed to those that have been created and not exported? if so is there a way to backup the plans? what are the implications for deleting these plans, are they searchable with the existing database?

briri commented 5 years ago

Hi @Ebrahim1010,

Plan PDFs are generated on demand by the system. The exported_plans table simply records the number of times a plan has been downloaded. You would just be deleting download counts for plans that are no longer in your database. The actual content of a plan is stored in the plans and answers tables

With that said though, its always good practice to backup your data prior to making large changes like this. If you're using MySQL you can use a command like the following to make a copy of your DB: mysqldump --single-transaction --no-create-db dmp -h[my.database.server] -p[my port] -u[my username] -p > /some/location/my_database.sql and then restore it (if needed) via: mysql dmp -h[my.database.server] -p[my port] -u[my username] -p < /some/location/my_database.sql

For more information on the database tables and their relationships with one another please refer to our data model diagram

norman-smith commented 5 years ago

Hi Sam

I’m having an issue finding these invalid records. Could you assist perhaps.

Checking Language records 0 records with a empty name field 0 records that are an invalid length for fields [:name] should be <= 20 0 records with a empty abbreviation field 0 records that are not unique per (abbreviation) 0 records that are an invalid length for fields [:abbreviation] should be <= 5 5 records that do not have valid [:abbreviation] 0 records that do not have a valid value for [:default_language], should be [true, false] Checking Note records 0 records with a empty text field 3 orphaned records due to nil or missing answer 0 orphaned records due to nil or missing user 0 records that do not have a valid value for [:archived], should be [true, false]

Regards Norman

From: Ebrahim1010 notifications@github.com Sent: Thursday, 11 April 2019 14:54 To: DMPRoadmap/roadmap roadmap@noreply.github.com Cc: Norman Smith norman.smith@uct.ac.za; Mention mention@noreply.github.com Subject: Re: [DMPRoadmap/roadmap] Template Migration fail (#2070)

Hi Sam,

Thank you for the help!

I wanted some clarification on the exported plans and destroying them. By destroying the exported plans does it mean that we are deleting the plans in the databse that have been created and exported as opposed to those that have been created and not exported? if so is there a way to backup the plans? what are the implications for deleting these plans, are they searchable with the existing database?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://protect-za.mimecast.com/s/ZWrcCzm4GXCvzP7mt4XfoG, or mute the threadhttps://protect-za.mimecast.com/s/Pbv9CAnX51iM5m81f8ae14.

Disclaimer - University of Cape Town This email is subject to UCT policies and email disclaimer published on our website at http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650 9111. If this email is not related to the business of UCT, it is sent by the sender in an individual capacity. Please report security incidents or abuse via https://csirt.uct.ac.za/page/report-an-incident.php.

xsrust commented 5 years ago

hi @norman-smith

Reading from your error message, you have Language : 5 records that do not have valid [:abbreviation] Note : 3 orphaned records due to nil or missing answer

You'll want to take the same approach for the notes detailed in my previous response: https://github.com/DMPRoadmap/roadmap/issues/2070#issuecomment-481669906 but this time searching for nonexisting answer reccords on the note, instead of nonexisting plans on the exportedPlan.

As there are no answers for this note, and it cannot be displayed, I would simply remove it from your system.

For your Languages, you should just need to follow the approach listed in my previous response. To check the validity of an object, call it's .valid? method, and to check it's errors, .errors.
The Rails page on validations might be helpful when interacting with your data: https://guides.rubyonrails.org/v5.2/active_record_validations.html#working-with-validation-errors

norman-smith commented 5 years ago

Hi Sam

Thanks for this. I’ve corrected all issues and the find_invalid_records displays 0 invalid records.

Regards Norman

From: Sam Rust notifications@github.com Sent: Tuesday, 16 April 2019 15:22 To: DMPRoadmap/roadmap roadmap@noreply.github.com Cc: Norman Smith norman.smith@uct.ac.za; Mention mention@noreply.github.com Subject: Re: [DMPRoadmap/roadmap] Template Migration fail (#2070)

hi @norman-smithhttps://protect-za.mimecast.com/s/4FWjC76y5Xt9rr1PC86RB7

Reading from your error message, you have Language : 5 records that do not have valid [:abbreviation] Note : 3 orphaned records due to nil or missing answer

You'll want to take the same approach for the notes detailed in my previous response:

2070 (comment)https://protect-za.mimecast.com/s/d3ZfC8qz5ESxmmpJu1XYWf

but this time searching for nonexisting answer reccords on the note, instead of nonexisting plans on the exportedPlan.

As there are no answers for this note, and it cannot be displayed, I would simply remove it from your system.

For your Languages, you should just need to follow the approach listed in my previous response. To check the validity of an object, call it's .valid? method, and to check it's errors, .errors. The Rails page on validations might be helpful when interacting with your data: https://guides.rubyonrails.org/v5.2/active_record_validations.html#working-with-validation-errorshttps://protect-za.mimecast.com/s/okT7C98A5gUPJJEZI30kt0

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://protect-za.mimecast.com/s/ZG9bC0go6LflnnwEFWO6Qx, or mute the threadhttps://protect-za.mimecast.com/s/oLlqCg5Dg2SnvvZgIE638L.

Disclaimer - University of Cape Town This email is subject to UCT policies and email disclaimer published on our website at http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650 9111. If this email is not related to the business of UCT, it is sent by the sender in an individual capacity. Please report security incidents or abuse via https://csirt.uct.ac.za/page/report-an-incident.php.

xsrust commented 5 years ago

That's excellent news! Provided your migration's now gone working, I'm going to close this ticket out, but if you run into any other problems, feel free to open a new issue for it.