escattone
commented
7 years ago First pass comments:
- Making the sitemaps failed initially (the command was being "killed" when running
./manage.py make_sitemapson the pod for the "web" deployment -- probably by linux for violating thecgroupsmemory limit for the "web" deployment). When we bumped up the memory limit of the "web" deployment to 8Gi, the command worked fine. The making of the sitemaps is normally a set of individual Celery tasks (one for each locale, and then a final one for the index) rather than a single task executed in one go, so there might not be any need to permanently raise the memory limit of the "celery-worker" deployment. - I had to create, populate, and promote a new search index.
- I had to run
./manage.py make_humanson the pod for the "web" deployment to get a successful response for https://mdn-mm.moz.works/humans.txt. - I had to set constance.KUMASCRIPT_TIMEOUT -- via the admin console -- to a non-zero value to get a successful response for https://mdn-mm.moz.works/media/kumascript-revision.txt.
- 404 for https://mdn-mm.moz.works/robots.txt. This is because it's being served from
MEDIA_ROOTwhich in AWS is/mdn/www, but we didn't copy the/app/media/robots.txtand/app/media/robots-go-away.txtfiles from the repo into that directory. I think I'll create a new Kuma PR that changes the service root fromMEDIA_ROOTto a newROBOTS_ROOTvalue, and then the default for that will be the value ofMEDIA_ROOT(SCL3) unless it's set from the environment (AWS). See https://github.com/mozilla/kuma/pull/4437. - the "celery-beat" pod generated the following error in maintenance mode (see https://sentry.prod.mozaws.net/operations/mdn-stage/issues/649650/):
DBAccessError: (13, 'Permission denied') File "celery/apps/beat.py", line 112, in start_scheduler beat.start() File "celery/beat.py", line 470, in start humanize_seconds(self.scheduler.max_interval)) File "kombu/utils/__init__.py", line 325, in __get__ value = obj.__dict__[self.__name__] = self.__get(obj) File "celery/beat.py", line 512, in scheduler return self.get_scheduler() File "celery/beat.py", line 507, in get_scheduler lazy=lazy) File "celery/utils/imports.py", line 53, in instantiate return symbol_by_name(name)(*args, **kwargs) File "celery/beat.py", line 358, in __init__ Scheduler.__init__(self, *args, **kwargs) File "celery/beat.py", line 185, in __init__ self.setup_schedule() File "celery/beat.py", line 384, in setup_schedule self._store = self._destroy_open_corrupted_schedule(exc) File "celery/beat.py", line 372, in _destroy_open_corrupted_schedule return self._open_schedule() File "celery/beat.py", line 366, in _open_schedule return self.persistence.open(self.schedule_filename, writeback=True) File "python2.7/shelve.py", line 243, in open return DbfilenameShelf(filename, flag, protocol, writeback) File "python2.7/shelve.py", line 227, in __init__ Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback) File "python2.7/anydbm.py", line 85, in open return mod.open(file, flag, mode) File "python2.7/dbhash.py", line 18, in open return bsddb.hashopen(file, flag, mode) File "bsddb/__init__.py", line 364, in hashopen d.open(file, db.DB_HASH, flags, mode)
bookshelfdave
Copied from PR #469:
This checklist uses these URLs:
Preparing for Maintenance Mode
If you want a public-facing site, you'll need to generate content before putting the site in Maintenance Mode. The full steps are in todo, but here's the quick version:
KUMA_WIKI_IFRAME_ALLOWED_HOSTSManual Sanity Check
Home page
Load https://mdn-mm.moz.works/en-US
Article page
Load https://mdn-mm.moz.works/en-US/docs/Web/HTML
Maintenance Mode page
Click a banner to load https://mdn-mm.moz.works/en-US/maintenance-mode
Automated Checks
With a Kuma environment, run the functional tests, using a command like:
These check that disabled endpoints redirect to the maintenance page, and that other pages are OK. It duplicates the Manual Sanity Check.
Full Manual Tests
Note: Many of these are candidates for headless testing
Background
Maintenance Mode is used to serve a recent copy of the site content, without allowing logins or database writes. It is used when the usual database is unavailable due to maintenance, such as a long-running migration or a datacenter transfer.
It is enabled with the environment variable
MAINTENANCE_MODE=True. More information and local development setup is in the Kuma documentation.It is possible, but not required, to run maintenance mode against a read-only database.
In this mode, no one is logged in and cookies are not sent. Articles can not be created, translated, or edited. If a page was not rendered, it can't be in maintenance mode, and KumaScript macros will be seen in the output.
Background tasks may still be scheduled, so the celery broker should remain available. Many tasks are completed with no action. Others, such as sitemap generation and cacheback refreshing, continue to run as normal.