Fix Docker build issues and refactor Fabfile

RealOrangeOne commented 1 year ago

Replaces https://github.com/wagtail/wagtail.org/pull/389

Remove the need for AWS / Heroku CLIs - this is now all done locally, outside the container / VM
Remove the ability to push data to production / staging - this is almost always a bad idea
Stop using a fork of fabric - use the host version
Remove make wrappers around fab commands
Automate deployment for branches

RealOrangeOne commented 1 year ago

My migrations don't seem to be sticking

This might happen depending on how you stop the project. Removing the make start shell should keep the data (it does for me). But if you manually run something like docker-compose down, that will likely remove it.

vossisboss commented 1 year ago

@RealOrangeOne I ran make start and then left it active. As far as I know, I'm not stopping the container at all while running the other commands. Definitely not using anything like docker-compose down. Just make migrate and make runserver.

vossisboss commented 1 year ago

P.S. I'll give your new commits a go some time after my meetings are done.

vossisboss commented 1 year ago

@RealOrangeOne I got the error No module named 'invoke' when running fab pull-production-data. So I added invoke and tried the command again.

The command ran but I didn't see the usual stream of pg_dump lines. I got this:

 ›   Warning: heroku update available from 8.1.7 to 8.1.9.
Getting backup from wagtail-org-production... fetching url of #425
Getting backup from wagtail-org-production... done, #425
CREATE SCHEMA
UPDATE 1
Default site's hostname was updated to 'localhost:8000'.
Any superuser accounts you previously created locally will have been wiped and will need to be recreated.

And then when I ran make runserver and navigated to localhost:8000, I got this error:

The above exception (relation "database_cache" does not exist
LINE 1: SELECT "cache_key", "value", "expires" FROM "database_cache"...
                                                    ^
) was the direct cause of the following exception:
  File "/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 204, in _get_response
    response = response.render()
  File "/venv/lib/python3.11/site-packages/django/template/response.py", line 105, in render
    self.content = self.rendered_content
  File "/venv/lib/python3.11/site-packages/django/template/response.py", line 83, in rendered_content
    return template.render(context, self._request)
  File "/venv/lib/python3.11/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 170, in render
    return self._render(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 162, in _render
    return self.nodelist.render(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 938, in render
    bit = node.render_annotated(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 905, in render_annotated
    return self.render(context)
  File "/venv/lib/python3.11/site-packages/pattern_library/loader_tags.py", line 38, in render
    return super().render(context)
  File "/venv/lib/python3.11/site-packages/django/template/loader_tags.py", line 150, in render
    return compiled_parent._render(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 162, in _render
    return self.nodelist.render(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 938, in render
    bit = node.render_annotated(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 905, in render_annotated
    return self.render(context)
  File "/venv/lib/python3.11/site-packages/pattern_library/loader_tags.py", line 38, in render
    return super().render(context)
  File "/venv/lib/python3.11/site-packages/django/template/loader_tags.py", line 150, in render
    return compiled_parent._render(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 162, in _render
    return self.nodelist.render(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 938, in render
    bit = node.render_annotated(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 905, in render_annotated
    return self.render(context)
  File "/venv/lib/python3.11/site-packages/django/template/loader_tags.py", line 62, in render
    result = block.nodelist.render(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 938, in render
    bit = node.render_annotated(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 905, in render_annotated
    return self.render(context)
  File "/venv/lib/python3.11/site-packages/django/template/defaulttags.py", line 315, in render
    return nodelist.render(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 938, in render
    bit = node.render_annotated(context)
  File "/venv/lib/python3.11/site-packages/django/template/base.py", line 905, in render_annotated
    return self.render(context)
  File "/venv/lib/python3.11/site-packages/django/template/library.py", line 192, in render
    output = self.func(*resolved_args, **resolved_kwargs)
  File "/venv/lib/python3.11/site-packages/wagtail/templatetags/wagtailcore_tags.py", line 47, in fullpageurl
    return page.get_full_url(request=context.get("request"))
  File "/venv/lib/python3.11/site-packages/wagtail/models/__init__.py", line 2123, in get_full_url
    url_parts = self.get_url_parts(request=request)
  File "/venv/lib/python3.11/site-packages/wagtail/models/__init__.py", line 2066, in get_url_parts
    possible_sites = self._get_relevant_site_root_paths(request)
  File "/venv/lib/python3.11/site-packages/wagtail/models/__init__.py", line 2044, in _get_relevant_site_root_paths
    for srp in self._get_site_root_paths(cache_object)
  File "/venv/lib/python3.11/site-packages/wagtail/models/__init__.py", line 2033, in _get_site_root_paths
    cache_object._wagtail_cached_site_root_paths = Site.get_site_root_paths()
  File "/venv/lib/python3.11/site-packages/wagtail/models/sites.py", line 212, in get_site_root_paths
    result = cache.get("wagtail_site_root_paths")
  File "/venv/lib/python3.11/site-packages/django/core/cache/backends/db.py", line 51, in get
    return self.get_many([key], version).get(key, default)
  File "/venv/lib/python3.11/site-packages/django/core/cache/backends/db.py", line 68, in get_many
    cursor.execute(
  File "/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 98, in execute
    return super().execute(sql, params)
  File "/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "/venv/lib/python3.11/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)

Exception Type: ProgrammingError at /
Exception Value: relation "database_cache" does not exist
LINE 1: SELECT "cache_key", "value", "expires" FROM "database_cache"...

I did run make migrate before pulling the production data in. Any ideas to correct this?

thibaudcolas commented 1 year ago

I’ve not tested this yet but I see this gets a thumbs up from Sage and the current setup is definitely not working at all, so will merge this now.

wagtail / wagtail.org

Fix Docker build issues and refactor Fabfile #392