Unpin pulpcore version and editable pulp_ansible

[chouseknecht]

Part of #238

[alikins]

@bmclaughlin I haven't tested yet, but I plan to test it today.

[alikins]

@cutwater @bmclaughlin Seems to mostly work for me. I have brought it up, but haven't applied the migrations yet.

[alikins]

@cutwater @bmclaughlin Tested this out and hit an error with migrations https://gist.github.com/alikins/a024e44de428a343c151165a042cd85b

perations to perform:
  Apply all migrations: admin, ansible, auth, contenttypes, core, sessions
Running migrations:
  Applying core.0013_repository_pulp_type...Traceback (most recent call last):
  File "/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.NotNullViolation: column "pulp_type" contains null values

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/venv/bin/django-admin", line 8, in <module>
    sys.exit(execute_from_command_line())
  File "/venv/lib64/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/venv/lib64/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/venv/lib64/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/venv/lib64/python3.6/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/venv/lib64/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/venv/lib64/python3.6/site-packages/django/core/management/commands/migrate.py", line 234, in handle
    fake_initial=fake_initial,
  File "/venv/lib64/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/venv/lib64/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/venv/lib64/python3.6/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/venv/lib64/python3.6/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/venv/lib64/python3.6/site-packages/django/db/migrations/operations/fields.py", line 112, in database_forwards
    field,
  File "/venv/lib64/python3.6/site-packages/django/db/backends/base/schema.py", line 447, in add_field
    self.execute(sql, params)
  File "/venv/lib64/python3.6/site-packages/django/db/backends/base/schema.py", line 137, in execute
    cursor.execute(sql, params)
  File "/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 99, in execute
    return super().execute(sql, params)
  File "/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/venv/lib64/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/venv/lib64/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: column "pulp_type" contains null values

make: *** [Makefile:51: docker/run-migrations] Error 1

[cutwater]

@alikins To restate what we discussed on call. This migration does not only provides a new field, it's breaking the migration process of existing database. For us it means we cannot apply it automatically and will have to apply it manually.

To apply this migration on existing database we will need to generate SQL and execute it as the following steps:

1. add pulp_type nullable field.
2. set appropriate value for pulp_type field for relevant repositories.
3. update pulp_type as not nullable.
4. update django_migrations table, insert record that 0013_repository_pulp_type is applied.

[alikins]

The manual migrations I ended up doing:

ALTER TABLE core_repository ADD COLUMN pulp_type text;
ALTER TABLE core_repository ALTER COLUMN pulp_type SET NOT NULL;
UPDATE core_repository SET pulp_type = 'ansible.ansible' WHERE pulp_id = '<the pulp_id of my ansible repo in core_repository>';

# NOTE: suspect this is not actually needed. I ended up doing it at
# some point because I typo'ed the migration name and hadn't applied
# what I thought I had. I think it is addressed in the migrations.
ALTER TABLE ansible_collectionversion ADD COLUMN is_certified bool;

#
# I may have 'manage migrate --fake pulp 0013_repository_pulp_type.py'  here
# and then the rest of the migrations
#
# Add a ansible_repo ptr to the core repo the right pulp_id_ptr
INSERT INTO ansible_ansiblerepository (repository_ptr_id)  VALUES ('<the pulp_id of my ansible repo in core_repository>');

[alikins]

@cutwater As mentioned in scrum, I agree the migrations probably would have went a lot further if 'pulp_type' column of 'core_repository' was made nullable in the migrations