search

dimagi / commcare-hq

CommCareHQ is the server backend for CommCare, the world's largest platform for designing, managing, and deploying robust, offline-first, mobile applications to frontline workers worldwide
https://www.dimagi.com/open-source/
BSD 3-Clause "New" or "Revised" License
497 stars 214 forks source link

manage.py migrate on new installation fails with corehq.doctypemigrations.djangomigrations.MigrationNotComplete #13744

Closed yuriiz closed 7 years ago

yuriiz commented 7 years ago

After following instructions from https://github.com/dimagi/commcare-hq/blob/master/README.md running

$ env CCHQ_IS_FRESH_INSTALL=1 ./manage.py migrate --noinput

Fails with:

(commcarehq) commcarehq@ip-172-30-2-68:~/commcare-hq$ ./manage.py migrate
Operations to perform:
  Synchronize unmigrated apps: ninetyninedots, mobile_auth, toggle, preindex, style, hqcouchlog, intrahealth, up_nrhm, hope, kookoo, pact, yo, hqpillow_retry, translations, util, megamobile, appstore, twilio, receiverwrapper, fluff, settings, prelogin, smsgh, common, couchapps, tropo, djangular, formtranslate, django_otp, statici18n, analytics, ws4redis, export, repeaters, a5288, care_pathways, staticfiles, importer, reports_core, bihar, enikshay, sislog, performance_sms, gunicorn, casegroups, mvp_docs, gsid, succeed, groups, hsph, mvp, styleguide, unicel, reports, fri, couch_sql_migration, preview_app, mocha, api, registration, hqcase, consumption, grapevine, custom_data_fields, ucla, langcodes, indicators, mach, mvp_indicators, mptt, crs_reports, djtables, compressor, auditcare, case, builds, mc, wisepill, uth, fluff_filter, dashboard, couchexport, world_vision, tabs, soil, utils, domainsync, pnlppgi, sql_db, couchlog, change_feed, openclinica, humanize, es, test, crispy_forms, http, users, hqmedia, crud, cachehq, hqscripts, callcenter, apposit, programs, cloudcare, django, ext, push
  Apply all migrations: logistics, domain, zipline, case_search, locations, phone, sites, couchforms, ivr, opm, accounting, fixtures, phonelog, doctypemigrations, sofabed, blobs, smsbillables, sms, otp_static, ilsgateway, django_digest, contenttypes, cleanup, commtrack, hqadmin, pillowtop, tzmigration, notifications, formplayer, icds_reports, sessions, ota, data_interfaces, auth, tastypie, form_processor, pillow_retry, data_analytics, dropbox, sql_proxy_accessors, django_prbac, userhack, sql_accessors, telerivet, captcha, admin, zapier, two_factor, m4change, djcelery, smsforms, ewsghana, tour, toggle_ui, products, userreports, otp_totp, hqwebapp, stock, reminders
Synchronizing apps without migrations:
2016-10-27 03:21:22,750 INFO synced 'auditcare' in couchdb
2016-10-27 03:21:22,797 INFO synced 'case' in couchdb
2016-10-27 03:21:22,841 INFO synced 'phone' in couchdb
2016-10-27 03:21:22,925 INFO synced 'phone' in couchdb
2016-10-27 03:21:22,981 INFO synced 'domain' in couchdb
2016-10-27 03:21:23,061 INFO synced 'domain' in couchdb
2016-10-27 03:21:23,145 INFO synced 'domain' in couchdb
2016-10-27 03:21:23,225 INFO synced 'domain' in couchdb
2016-10-27 03:21:23,273 INFO synced 'hqadmin' in couchdb
2016-10-27 03:21:23,329 INFO synced 'hqmedia' in couchdb
2016-10-27 03:21:23,377 INFO synced 'consumption' in couchdb
2016-10-27 03:21:23,425 INFO synced 'couchforms' in couchdb
2016-10-27 03:21:23,473 INFO synced 'couchexport' in couchdb
2016-10-27 03:21:23,529 INFO synced 'couchlog' in couchdb
2016-10-27 03:21:23,573 INFO synced 'custom_data_fields' in couchdb
2016-10-27 03:21:23,617 INFO synced 'app_manager' in couchdb
2016-10-27 03:21:23,661 INFO synced 'fixtures' in couchdb
2016-10-27 03:21:23,709 INFO synced 'reminders' in couchdb
2016-10-27 03:21:23,757 INFO synced 'translations' in couchdb
2016-10-27 03:21:23,806 INFO synced 'users' in couchdb
2016-10-27 03:21:23,853 INFO synced 'groups' in couchdb
2016-10-27 03:21:23,901 INFO synced 'mobile_auth' in couchdb
2016-10-27 03:21:23,945 INFO synced 'performance_sms' in couchdb
2016-10-27 03:21:23,993 INFO synced 'registration' in couchdb
2016-10-27 03:21:24,041 INFO synced 'userreports' in couchdb
2016-10-27 03:21:24,097 INFO synced 'builds' in couchdb
2016-10-27 03:21:24,145 INFO synced 'indicators' in couchdb
2016-10-27 03:21:24,257 INFO synced 'phone_numbers' in couchdb
2016-10-27 03:21:24,341 INFO synced 'reportconfig' in couchdb
2016-10-27 03:21:24,425 INFO synced 'case_by_domain_hq_user_id_type' in couchdb
2016-10-27 03:21:24,509 INFO synced 'program_by_code' in couchdb
2016-10-27 03:21:24,593 INFO synced 'cases_by_domain_external_id' in couchdb
2016-10-27 03:21:24,673 INFO synced 'export_instances_by_domain' in couchdb
2016-10-27 03:21:24,757 INFO synced 'all_forms' in couchdb
2016-10-27 03:21:24,837 INFO synced 'deleted_users_by_username' in couchdb
2016-10-27 03:21:24,921 INFO synced 'all_case_properties' in couchdb
2016-10-27 03:21:25,005 INFO synced 'doc_conflicts' in couchdb
2016-10-27 03:21:25,089 INFO synced 'form_case_index' in couchdb
2016-10-27 03:21:25,173 INFO synced 'open_cases' in couchdb
2016-10-27 03:21:25,253 INFO synced 'apps_with_submissions' in couchdb
2016-10-27 03:21:25,333 INFO synced 'app_translations_by_popularity' in couchdb
2016-10-27 03:21:25,417 INFO synced 'app_translations_by_popularity' in couchdb
2016-10-27 03:21:25,497 INFO synced 'exports_forms_by_app' in couchdb
2016-10-27 03:21:25,581 INFO synced 'exports_forms_by_app' in couchdb
2016-10-27 03:21:25,665 INFO synced 'attachments' in couchdb
2016-10-27 03:21:25,749 INFO synced 'exports_forms_by_xform' in couchdb
2016-10-27 03:21:25,833 INFO synced 'cases_by_server_date' in couchdb
2016-10-27 03:21:25,917 INFO synced 'schemas_by_xmlns_or_case_type' in couchdb
2016-10-27 03:21:26,001 INFO synced 'cases_get_lite' in couchdb
2016-10-27 03:21:26,085 INFO synced 'deleted_data' in couchdb
2016-10-27 03:21:26,165 INFO synced 'receiverwrapper' in couchdb
2016-10-27 03:21:26,249 INFO synced 'case_types_by_domain' in couchdb
2016-10-27 03:21:26,333 INFO synced 'supply_point_by_loc' in couchdb
2016-10-27 03:21:26,417 INFO synced 'form_question_schema' in couchdb
2016-10-27 03:21:26,501 INFO synced 'cases_by_owner' in couchdb
2016-10-27 03:21:26,581 INFO synced 'users_extra' in couchdb
2016-10-27 03:21:26,661 INFO synced 'forms_by_app_info' in couchdb
2016-10-27 03:21:26,745 INFO synced 'forms_by_app_info' in couchdb
2016-10-27 03:21:26,829 INFO synced 'case_indices' in couchdb
2016-10-27 03:21:26,910 INFO synced 'sync_logs_by_date' in couchdb
2016-10-27 03:21:26,993 INFO synced 'tests' in couchdb
2016-10-27 03:21:27,073 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:21:27,153 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:21:27,237 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:21:27,317 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:21:27,397 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:21:27,477 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:21:27,557 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:21:27,637 INFO synced 'inferred_schemas_by_case_type' in couchdb
2016-10-27 03:21:27,717 INFO synced 'export_instances_by_is_daily_saved' in couchdb
2016-10-27 03:21:27,797 INFO synced 'all_docs' in couchdb
2016-10-27 03:21:27,877 INFO synced 'all_docs' in couchdb
2016-10-27 03:21:27,965 INFO synced 'all_docs' in couchdb
2016-10-27 03:21:28,049 INFO synced 'all_docs' in couchdb
2016-10-27 03:21:28,129 INFO synced 'all_docs' in couchdb
2016-10-27 03:21:28,209 INFO synced 'all_docs' in couchdb
2016-10-27 03:21:28,257 INFO synced 'wisepill' in couchdb
2016-10-27 03:21:28,305 INFO synced 'fri' in couchdb
2016-10-27 03:21:28,366 INFO synced 'fluff' in couchdb
2016-10-27 03:21:28,449 INFO synced 'fluff' in couchdb
2016-10-27 03:21:28,529 INFO synced 'fluff' in couchdb
2016-10-27 03:21:28,581 INFO synced 'fluff_filter' in couchdb
2016-10-27 03:21:28,637 INFO synced 'mvp_births_indicators' in couchdb
2016-10-27 03:21:28,721 INFO synced 'mvp_deaths_indicators' in couchdb
2016-10-27 03:21:28,809 INFO synced 'mvp_over5_indicators' in couchdb
2016-10-27 03:21:28,893 INFO synced 'mvp_chw_visits_indicators' in couchdb
2016-10-27 03:21:28,977 INFO synced 'mvp_verbal_autopsy_indicators' in couchdb
2016-10-27 03:21:29,061 INFO synced 'mvp_maternal_health_indicators' in couchdb
2016-10-27 03:21:29,145 INFO synced 'mvp_chw_referrals_indicators' in couchdb
2016-10-27 03:21:29,229 INFO synced 'mvp_child_health_indicators' in couchdb
2016-10-27 03:21:29,277 INFO synced 'pact' in couchdb
2016-10-27 03:21:29,325 INFO synced 'm4change' in couchdb
2016-10-27 03:21:29,373 INFO synced 'uth' in couchdb
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying doctypemigrations.0004_auto_20151001_1809...Traceback (most recent call last):
  File "./manage.py", line 95, in <module>
    execute_from_command_line(sys.argv)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 222, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 148, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/db/migrations/migration.py", line 115, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/home/commcarehq/commcare-hq/corehq/sql_db/operations.py", line 27, in database_forwards
    super(HqOpMixin, self).database_forwards(app_label, schema_editor, from_state, to_state)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/db/migrations/operations/special.py", line 183, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/home/commcarehq/commcare-hq/corehq/doctypemigrations/djangomigrations.py", line 43, in forwards
    raise MigrationNotComplete(MIGRATION_MESSAGE.format(slug=migrator.slug))
corehq.doctypemigrations.djangomigrations.MigrationNotComplete: 
Before you can merge you must run the user_db_migration doc_type migration.

If you're seeing this on your **dev machine**, run

./manage.py sync_couch_views
./manage.py run_doctype_migration user_db_migration --initial
./manage.py migrate

and that final migrate should go through without a hitch.

After you migrate/update your code and everything's working fine, run

./manage.py run_doctype_migration user_db_migration --cleanup

to delete the documents from the source db that have been copied to the target db

If you're seeing this on a **production deploy**
take note of the full file path to this aborted release,
something like "/home/cchq/www/production/releases/2015-10-16_19.37",
and then follow the full instructions for the zero-downtime live swap:

https://github.com/dimagi/commcare-hq/blob/master/corehq/doctypemigrations/README.md#run-the-doctype-migration

Instead of the "current" release, use the path to this release.
(This simply has the effect of running the process on up-to-date code.)

but running

./manage.py sync_couch_views
./manage.py run_doctype_migration user_db_migration --initial
./manage.py migrate

does not fix the issue:

(commcarehq) commcarehq@ip-172-30-2-68:~/commcare-hq$ ./manage.py sync_couch_views
Syncing design docs for app_manager (using ExtraPreindexPlugin)
2016-10-27 03:23:06,179 INFO synced 'app_manager' in couchdb
Syncing design docs for auditcare
2016-10-27 03:23:06,231 INFO synced 'auditcare' in couchdb
Syncing design docs for builds
2016-10-27 03:23:06,283 INFO synced 'builds' in couchdb
Syncing design docs for case
2016-10-27 03:23:06,339 INFO synced 'case' in couchdb
Syncing design docs for consumption
2016-10-27 03:23:06,391 INFO synced 'consumption' in couchdb
Syncing design docs for couchapps (using CouchAppsPreindexPlugin)
2016-10-27 03:23:06,571 INFO synced 'phone_numbers' in couchdb
2016-10-27 03:23:06,663 INFO synced 'reportconfig' in couchdb
2016-10-27 03:23:06,755 INFO synced 'case_by_domain_hq_user_id_type' in couchdb
2016-10-27 03:23:06,847 INFO synced 'program_by_code' in couchdb
2016-10-27 03:23:06,939 INFO synced 'cases_by_domain_external_id' in couchdb
2016-10-27 03:23:07,027 INFO synced 'export_instances_by_domain' in couchdb
2016-10-27 03:23:07,119 INFO synced 'all_forms' in couchdb
2016-10-27 03:23:07,207 INFO synced 'deleted_users_by_username' in couchdb
2016-10-27 03:23:07,299 INFO synced 'all_case_properties' in couchdb
2016-10-27 03:23:07,391 INFO synced 'doc_conflicts' in couchdb
2016-10-27 03:23:07,483 INFO synced 'form_case_index' in couchdb
2016-10-27 03:23:07,575 INFO synced 'open_cases' in couchdb
2016-10-27 03:23:07,663 INFO synced 'apps_with_submissions' in couchdb
2016-10-27 03:23:07,751 INFO synced 'app_translations_by_popularity' in couchdb
2016-10-27 03:23:07,843 INFO synced 'app_translations_by_popularity' in couchdb
2016-10-27 03:23:07,931 INFO synced 'exports_forms_by_app' in couchdb
2016-10-27 03:23:08,023 INFO synced 'exports_forms_by_app' in couchdb
2016-10-27 03:23:08,115 INFO synced 'attachments' in couchdb
2016-10-27 03:23:08,207 INFO synced 'exports_forms_by_xform' in couchdb
2016-10-27 03:23:08,303 INFO synced 'cases_by_server_date' in couchdb
2016-10-27 03:23:08,391 INFO synced 'schemas_by_xmlns_or_case_type' in couchdb
2016-10-27 03:23:08,483 INFO synced 'cases_get_lite' in couchdb
2016-10-27 03:23:08,575 INFO synced 'deleted_data' in couchdb
2016-10-27 03:23:08,663 INFO synced 'receiverwrapper' in couchdb
2016-10-27 03:23:08,755 INFO synced 'case_types_by_domain' in couchdb
2016-10-27 03:23:08,847 INFO synced 'supply_point_by_loc' in couchdb
2016-10-27 03:23:08,935 INFO synced 'form_question_schema' in couchdb
2016-10-27 03:23:09,027 INFO synced 'cases_by_owner' in couchdb
2016-10-27 03:23:09,115 INFO synced 'users_extra' in couchdb
2016-10-27 03:23:09,203 INFO synced 'forms_by_app_info' in couchdb
2016-10-27 03:23:09,295 INFO synced 'forms_by_app_info' in couchdb
2016-10-27 03:23:09,387 INFO synced 'case_indices' in couchdb
2016-10-27 03:23:09,475 INFO synced 'sync_logs_by_date' in couchdb
2016-10-27 03:23:09,567 INFO synced 'tests' in couchdb
2016-10-27 03:23:09,655 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:09,743 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:09,835 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:09,923 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:10,011 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:10,099 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:10,187 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:10,275 INFO synced 'inferred_schemas_by_case_type' in couchdb
2016-10-27 03:23:10,363 INFO synced 'export_instances_by_is_daily_saved' in couchdb
2016-10-27 03:23:10,451 INFO synced 'all_docs' in couchdb
2016-10-27 03:23:10,539 INFO synced 'all_docs' in couchdb
2016-10-27 03:23:10,635 INFO synced 'all_docs' in couchdb
2016-10-27 03:23:10,723 INFO synced 'all_docs' in couchdb
2016-10-27 03:23:10,811 INFO synced 'all_docs' in couchdb
2016-10-27 03:23:10,899 INFO synced 'all_docs' in couchdb
Syncing design docs for couchexport
2016-10-27 03:23:10,955 INFO synced 'couchexport' in couchdb
Syncing design docs for couchforms
2016-10-27 03:23:11,007 INFO synced 'couchforms' in couchdb
Syncing design docs for couchlog
2016-10-27 03:23:11,063 INFO synced 'couchlog' in couchdb
Syncing design docs for custom_data_fields
2016-10-27 03:23:11,119 INFO synced 'custom_data_fields' in couchdb
Syncing design docs for domain (using ExtraPreindexPlugin)
2016-10-27 03:23:11,179 INFO synced 'domain' in couchdb
2016-10-27 03:23:11,267 INFO synced 'domain' in couchdb
2016-10-27 03:23:11,355 INFO synced 'domain' in couchdb
2016-10-27 03:23:11,443 INFO synced 'domain' in couchdb
Syncing design docs for fixtures (using ExtraPreindexPlugin)
2016-10-27 03:23:11,495 INFO synced 'fixtures' in couchdb
Syncing design docs for fluff (using FluffPreindexPlugin)
2016-10-27 03:23:11,571 INFO synced 'fluff' in couchdb
2016-10-27 03:23:11,659 INFO synced 'fluff' in couchdb
2016-10-27 03:23:11,747 INFO synced 'fluff' in couchdb
Syncing design docs for fluff_filter
2016-10-27 03:23:11,799 INFO synced 'fluff_filter' in couchdb
Syncing design docs for fri
2016-10-27 03:23:11,855 INFO synced 'fri' in couchdb
Syncing design docs for groups (using ExtraPreindexPlugin)
2016-10-27 03:23:11,907 INFO synced 'groups' in couchdb
Syncing design docs for hqadmin
2016-10-27 03:23:11,959 INFO synced 'hqadmin' in couchdb
Syncing design docs for hqmedia
2016-10-27 03:23:12,015 INFO synced 'hqmedia' in couchdb
Syncing design docs for indicators
2016-10-27 03:23:12,071 INFO synced 'indicators' in couchdb
Syncing design docs for m4change
2016-10-27 03:23:12,123 INFO synced 'm4change' in couchdb
Syncing design docs for mobile_auth
2016-10-27 03:23:12,175 INFO synced 'mobile_auth' in couchdb
Syncing design docs for mvp_indicators (using CouchAppsPreindexPlugin)
2016-10-27 03:23:12,247 INFO synced 'mvp_births_indicators' in couchdb
2016-10-27 03:23:12,335 INFO synced 'mvp_deaths_indicators' in couchdb
2016-10-27 03:23:12,423 INFO synced 'mvp_over5_indicators' in couchdb
2016-10-27 03:23:12,511 INFO synced 'mvp_chw_visits_indicators' in couchdb
2016-10-27 03:23:12,603 INFO synced 'mvp_verbal_autopsy_indicators' in couchdb
2016-10-27 03:23:12,691 INFO synced 'mvp_maternal_health_indicators' in couchdb
2016-10-27 03:23:12,779 INFO synced 'mvp_chw_referrals_indicators' in couchdb
2016-10-27 03:23:12,871 INFO synced 'mvp_child_health_indicators' in couchdb
Syncing design docs for pact
2016-10-27 03:23:12,927 INFO synced 'pact' in couchdb
Syncing design docs for performance_sms
2016-10-27 03:23:12,979 INFO synced 'performance_sms' in couchdb
Syncing design docs for phone (using ExtraPreindexPlugin)
2016-10-27 03:23:13,035 INFO synced 'phone' in couchdb
2016-10-27 03:23:13,127 INFO synced 'phone' in couchdb
Syncing design docs for registration
2016-10-27 03:23:13,179 INFO synced 'registration' in couchdb
Syncing design docs for reminders
2016-10-27 03:23:13,235 INFO synced 'reminders' in couchdb
Syncing design docs for translations
2016-10-27 03:23:13,295 INFO synced 'translations' in couchdb
Syncing design docs for userreports
2016-10-27 03:23:13,347 INFO synced 'userreports' in couchdb
Syncing design docs for users (using ExtraPreindexPlugin)
2016-10-27 03:23:13,399 INFO synced 'users' in couchdb
Syncing design docs for uth
2016-10-27 03:23:13,455 INFO synced 'uth' in couchdb
Syncing design docs for wisepill
2016-10-27 03:23:13,511 INFO synced 'wisepill' in couchdb
(commcarehq) commcarehq@ip-172-30-2-68:~/commcare-hq$ ./manage.py run_doctype_migration user_db_migration --initial
CommandError: You have already run an initial migration.
Run

./manage.py run_doctype_migration user_db_migration --continuous

for continuous replication starting where you left off.
(commcarehq) commcarehq@ip-172-30-2-68:~/commcare-hq$ ./manage.py migrate
Operations to perform:
  Synchronize unmigrated apps: ninetyninedots, mobile_auth, toggle, preindex, style, hqcouchlog, intrahealth, up_nrhm, hope, kookoo, pact, yo, hqpillow_retry, translations, util, megamobile, appstore, twilio, receiverwrapper, fluff, settings, prelogin, smsgh, common, couchapps, tropo, djangular, formtranslate, django_otp, statici18n, analytics, ws4redis, export, repeaters, a5288, care_pathways, staticfiles, importer, reports_core, bihar, enikshay, sislog, performance_sms, gunicorn, casegroups, mvp_docs, gsid, succeed, groups, hsph, mvp, styleguide, unicel, reports, fri, couch_sql_migration, preview_app, mocha, api, registration, hqcase, consumption, grapevine, custom_data_fields, ucla, langcodes, indicators, mach, mvp_indicators, mptt, crs_reports, djtables, compressor, auditcare, case, builds, mc, wisepill, uth, fluff_filter, dashboard, couchexport, world_vision, tabs, soil, utils, domainsync, pnlppgi, sql_db, couchlog, change_feed, openclinica, humanize, es, test, crispy_forms, http, users, hqmedia, crud, cachehq, hqscripts, callcenter, apposit, programs, cloudcare, django, ext, push
  Apply all migrations: logistics, domain, zipline, case_search, locations, phone, sites, couchforms, ivr, opm, accounting, fixtures, phonelog, doctypemigrations, sofabed, blobs, smsbillables, sms, otp_static, ilsgateway, django_digest, contenttypes, cleanup, commtrack, hqadmin, pillowtop, tzmigration, notifications, formplayer, icds_reports, sessions, ota, data_interfaces, auth, tastypie, form_processor, pillow_retry, data_analytics, dropbox, sql_proxy_accessors, django_prbac, userhack, sql_accessors, telerivet, captcha, admin, zapier, two_factor, m4change, djcelery, smsforms, ewsghana, tour, toggle_ui, products, userreports, otp_totp, hqwebapp, stock, reminders
Synchronizing apps without migrations:
2016-10-27 03:23:19,657 INFO synced 'auditcare' in couchdb
2016-10-27 03:23:19,705 INFO synced 'case' in couchdb
2016-10-27 03:23:19,753 INFO synced 'phone' in couchdb
2016-10-27 03:23:19,837 INFO synced 'phone' in couchdb
2016-10-27 03:23:19,889 INFO synced 'domain' in couchdb
2016-10-27 03:23:19,969 INFO synced 'domain' in couchdb
2016-10-27 03:23:20,053 INFO synced 'domain' in couchdb
2016-10-27 03:23:20,133 INFO synced 'domain' in couchdb
2016-10-27 03:23:20,181 INFO synced 'hqadmin' in couchdb
2016-10-27 03:23:20,229 INFO synced 'hqmedia' in couchdb
2016-10-27 03:23:20,281 INFO synced 'consumption' in couchdb
2016-10-27 03:23:20,329 INFO synced 'couchforms' in couchdb
2016-10-27 03:23:20,377 INFO synced 'couchexport' in couchdb
2016-10-27 03:23:20,421 INFO synced 'couchlog' in couchdb
2016-10-27 03:23:20,465 INFO synced 'custom_data_fields' in couchdb
2016-10-27 03:23:20,509 INFO synced 'app_manager' in couchdb
2016-10-27 03:23:20,553 INFO synced 'fixtures' in couchdb
2016-10-27 03:23:20,601 INFO synced 'reminders' in couchdb
2016-10-27 03:23:20,649 INFO synced 'translations' in couchdb
2016-10-27 03:23:20,697 INFO synced 'users' in couchdb
2016-10-27 03:23:20,745 INFO synced 'groups' in couchdb
2016-10-27 03:23:20,793 INFO synced 'mobile_auth' in couchdb
2016-10-27 03:23:20,837 INFO synced 'performance_sms' in couchdb
2016-10-27 03:23:20,885 INFO synced 'registration' in couchdb
2016-10-27 03:23:20,929 INFO synced 'userreports' in couchdb
2016-10-27 03:23:20,981 INFO synced 'builds' in couchdb
2016-10-27 03:23:21,029 INFO synced 'indicators' in couchdb
2016-10-27 03:23:21,137 INFO synced 'phone_numbers' in couchdb
2016-10-27 03:23:21,221 INFO synced 'reportconfig' in couchdb
2016-10-27 03:23:21,305 INFO synced 'case_by_domain_hq_user_id_type' in couchdb
2016-10-27 03:23:21,389 INFO synced 'program_by_code' in couchdb
2016-10-27 03:23:21,477 INFO synced 'cases_by_domain_external_id' in couchdb
2016-10-27 03:23:21,565 INFO synced 'export_instances_by_domain' in couchdb
2016-10-27 03:23:21,649 INFO synced 'all_forms' in couchdb
2016-10-27 03:23:21,729 INFO synced 'deleted_users_by_username' in couchdb
2016-10-27 03:23:21,813 INFO synced 'all_case_properties' in couchdb
2016-10-27 03:23:21,897 INFO synced 'doc_conflicts' in couchdb
2016-10-27 03:23:21,981 INFO synced 'form_case_index' in couchdb
2016-10-27 03:23:22,065 INFO synced 'open_cases' in couchdb
2016-10-27 03:23:22,145 INFO synced 'apps_with_submissions' in couchdb
2016-10-27 03:23:22,225 INFO synced 'app_translations_by_popularity' in couchdb
2016-10-27 03:23:22,309 INFO synced 'app_translations_by_popularity' in couchdb
2016-10-27 03:23:22,389 INFO synced 'exports_forms_by_app' in couchdb
2016-10-27 03:23:22,473 INFO synced 'exports_forms_by_app' in couchdb
2016-10-27 03:23:22,557 INFO synced 'attachments' in couchdb
2016-10-27 03:23:22,641 INFO synced 'exports_forms_by_xform' in couchdb
2016-10-27 03:23:22,725 INFO synced 'cases_by_server_date' in couchdb
2016-10-27 03:23:22,805 INFO synced 'schemas_by_xmlns_or_case_type' in couchdb
2016-10-27 03:23:22,889 INFO synced 'cases_get_lite' in couchdb
2016-10-27 03:23:22,973 INFO synced 'deleted_data' in couchdb
2016-10-27 03:23:23,053 INFO synced 'receiverwrapper' in couchdb
2016-10-27 03:23:23,137 INFO synced 'case_types_by_domain' in couchdb
2016-10-27 03:23:23,221 INFO synced 'supply_point_by_loc' in couchdb
2016-10-27 03:23:23,305 INFO synced 'form_question_schema' in couchdb
2016-10-27 03:23:23,389 INFO synced 'cases_by_owner' in couchdb
2016-10-27 03:23:23,469 INFO synced 'users_extra' in couchdb
2016-10-27 03:23:23,549 INFO synced 'forms_by_app_info' in couchdb
2016-10-27 03:23:23,633 INFO synced 'forms_by_app_info' in couchdb
2016-10-27 03:23:23,717 INFO synced 'case_indices' in couchdb
2016-10-27 03:23:23,797 INFO synced 'sync_logs_by_date' in couchdb
2016-10-27 03:23:23,881 INFO synced 'tests' in couchdb
2016-10-27 03:23:23,961 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:24,041 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:24,125 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:24,205 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:24,289 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:24,369 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:24,449 INFO synced 'by_domain_doc_type_date' in couchdb
2016-10-27 03:23:24,533 INFO synced 'inferred_schemas_by_case_type' in couchdb
2016-10-27 03:23:24,613 INFO synced 'export_instances_by_is_daily_saved' in couchdb
2016-10-27 03:23:24,693 INFO synced 'all_docs' in couchdb
2016-10-27 03:23:24,773 INFO synced 'all_docs' in couchdb
2016-10-27 03:23:24,857 INFO synced 'all_docs' in couchdb
2016-10-27 03:23:24,937 INFO synced 'all_docs' in couchdb
2016-10-27 03:23:25,017 INFO synced 'all_docs' in couchdb
2016-10-27 03:23:25,097 INFO synced 'all_docs' in couchdb
2016-10-27 03:23:25,145 INFO synced 'wisepill' in couchdb
2016-10-27 03:23:25,201 INFO synced 'fri' in couchdb
2016-10-27 03:23:25,261 INFO synced 'fluff' in couchdb
2016-10-27 03:23:25,341 INFO synced 'fluff' in couchdb
2016-10-27 03:23:25,421 INFO synced 'fluff' in couchdb
2016-10-27 03:23:25,469 INFO synced 'fluff_filter' in couchdb
2016-10-27 03:23:25,525 INFO synced 'mvp_births_indicators' in couchdb
2016-10-27 03:23:25,609 INFO synced 'mvp_deaths_indicators' in couchdb
2016-10-27 03:23:25,693 INFO synced 'mvp_over5_indicators' in couchdb
2016-10-27 03:23:25,777 INFO synced 'mvp_chw_visits_indicators' in couchdb
2016-10-27 03:23:25,861 INFO synced 'mvp_verbal_autopsy_indicators' in couchdb
2016-10-27 03:23:25,945 INFO synced 'mvp_maternal_health_indicators' in couchdb
2016-10-27 03:23:26,029 INFO synced 'mvp_chw_referrals_indicators' in couchdb
2016-10-27 03:23:26,113 INFO synced 'mvp_child_health_indicators' in couchdb
2016-10-27 03:23:26,161 INFO synced 'pact' in couchdb
2016-10-27 03:23:26,205 INFO synced 'm4change' in couchdb
2016-10-27 03:23:26,257 INFO synced 'uth' in couchdb
  Creating tables...
    Running deferred SQL...
  Installing custom SQL...
Running migrations:
  Rendering model states... DONE
  Applying doctypemigrations.0005_auto_20151013_0819...Traceback (most recent call last):
  File "./manage.py", line 95, in <module>
    execute_from_command_line(sys.argv)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute_from_command_line
    utility.execute()
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 346, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/core/management/base.py", line 394, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/core/management/base.py", line 445, in execute
    output = self.handle(*args, **options)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 222, in handle
    executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 110, in migrate
    self.apply_migration(states[migration], migration, fake=fake, fake_initial=fake_initial)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 148, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/db/migrations/migration.py", line 115, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/home/commcarehq/commcare-hq/corehq/sql_db/operations.py", line 27, in database_forwards
    super(HqOpMixin, self).database_forwards(app_label, schema_editor, from_state, to_state)
  File "/home/commcarehq/local/lib/python2.7/site-packages/django/db/migrations/operations/special.py", line 183, in database_forwards
    self.code(from_state.apps, schema_editor)
  File "/home/commcarehq/commcare-hq/corehq/doctypemigrations/djangomigrations.py", line 43, in forwards
    raise MigrationNotComplete(MIGRATION_MESSAGE.format(slug=migrator.slug))
corehq.doctypemigrations.djangomigrations.MigrationNotComplete: 
Before you can merge you must run the fixtures doc_type migration.

If you're seeing this on your **dev machine**, run

./manage.py sync_couch_views
./manage.py run_doctype_migration fixtures --initial
./manage.py migrate

and that final migrate should go through without a hitch.

After you migrate/update your code and everything's working fine, run

./manage.py run_doctype_migration fixtures --cleanup

to delete the documents from the source db that have been copied to the target db

If you're seeing this on a **production deploy**
take note of the full file path to this aborted release,
something like "/home/cchq/www/production/releases/2015-10-16_19.37",
and then follow the full instructions for the zero-downtime live swap:

https://github.com/dimagi/commcare-hq/blob/master/corehq/doctypemigrations/README.md#run-the-doctype-migration

Instead of the "current" release, use the path to this release.
(This simply has the effect of running the process on up-to-date code.)

(commcarehq) commcarehq@ip-172-30-2-68:~/commcare-hq$
benrudolph commented 7 years ago

Hi @yuriiz,

Based on what you've pasted, you actually did not run env CCHQ_IS_FRESH_INSTALL=1 ./manage.py migrate --noinput, but rather ./manage.py migrate. Also, if you look closely those two error messages are slightly different.

Setting up a dev environment locally can be tough. I'd also recommend taking a look at our docker setup: https://github.com/dimagi/commcare-hq/tree/master/docker

yuriiz commented 7 years ago

Yes, running env CCHQ_IS_FRESH_INSTALL=1 ./manage.py migrate --noinput again made this migration pass. Perhaps exception description could be changed to be not misleading.

benrudolph commented 7 years ago

Glad it worked out. Agree the message could be better