search

nautobot / nautobot-docker-compose

Docker Compose for Nautobot
47 stars 43 forks source link

Postgress and Celery beat service produce errors in log during the first deployment (when no db exists already). #47

Closed itheodoridis closed 2 months ago

itheodoridis commented 2 years ago

Postgress and Celery beat service produce errors in log during the first deployment (when no db exists already). Postgress complains for some relations that don't exist at specific character locations, e.g. :

nautobot-docker-postgres-1         | 2022-09-05 00:02:28.816 EEST [83] ERROR:  relation "django_content_type" does not exist at character 106
nautobot-docker-postgres-1         | 2022-09-05 00:02:28.816 EEST [83] STATEMENT:  SELECT "django_content_type"."id", "django_content_type"."app_label", "django_content_type"."model" FROM "django_content_type" WHERE (("django_content_type"."app_label" = 'extras' AND "django_content_type"."model" IN ('relationshipassociation', 'status', 'tag', 'dynamicgroup', 'configcontextschema', 'secret', 'secretsgroup')) OR ("django_content_type"."app_label" = 'dcim' AND "django_content_type"."model" IN ('consoleport', 'consoleserverport', 'powerport', 'poweroutlet', 'interface', 'frontport', 'rearport', 'devicebay', 'inventoryitem', 'manufacturer', 'devicetype', 'devicerole', 'platform', 'device', 'virtualchassis', 'cable', 'consoleporttemplate', 'consoleserverporttemplate', 'powerporttemplate', 'poweroutlettemplate', 'interfacetemplate', 'frontporttemplate', 'rearporttemplate', 'devicebaytemplate', 'locationtype', 'location', 'powerpanel', 'powerfeed', 'rackgroup', 'rackrole', 'rack', 'rackreservation', 'region', 'site')) OR ("django_content_type"."app_label" = 'circuits' AND "django_content_type"."model" IN ('providernetwork', 'provider', 'circuittype', 'circuit', 'circuittermination')) OR ("django_content_type"."app_label" = 'virtualization' AND "django_content_type"."model" IN ('clustertype', 'clustergroup', 'cluster', 'virtualmachine', 'vminterface')) OR ("django_content_type"."app_label" = 'ipam' AND "django_content_type"."model" IN ('vrf', 'routetarget', 'rir', 'aggregate', 'role', 'prefix', 'ipaddress', 'vlangroup', 'vlan', 'service')) OR ("django_content_type"."app_label" = 'tenancy' AND "django_content_type"."model" IN ('tenantgroup', 'tenant')))

Celery beat is producing python errors in the log, its quite possible that the two issues are connected. 

nautobot-docker-celery_beat-1      | celery beat v5.2.7 (dawn-chorus) is starting.
nautobot-docker-nautobot-1         |   Applying extras.0001_initial_part_1... OK
nautobot-docker-nautobot-1         |   Applying tenancy.0001_initial... OK
nautobot-docker-nautobot-worker-1  | 00:02:34.526 INFO    rq.worker :
nautobot-docker-nautobot-worker-1  |   Worker rq:worker:2143c5e82ac149eab2b7dbdb1d791d69: started, version 1.10.1
nautobot-docker-nautobot-worker-1  | 00:02:34.526 INFO    rq.worker :
nautobot-docker-nautobot-worker-1  |   Subscribing to channel rq:pubsub:2143c5e82ac149eab2b7dbdb1d791d69
nautobot-docker-nautobot-worker-1  | 00:02:34.534 INFO    rq.worker :
nautobot-docker-nautobot-worker-1  |   *** Listening on default, check_releases, custom_fields, webhooks...
nautobot-docker-nautobot-worker-1  | 00:02:34.536 INFO    rq.worker :
nautobot-docker-nautobot-worker-1  |   Cleaning registries for queue: default
nautobot-docker-nautobot-worker-1  | 00:02:34.540 INFO    rq.worker :
nautobot-docker-nautobot-worker-1  |   Cleaning registries for queue: check_releases
nautobot-docker-nautobot-worker-1  | 00:02:34.549 INFO    rq.worker :
nautobot-docker-nautobot-worker-1  |   Cleaning registries for queue: custom_fields
nautobot-docker-nautobot-worker-1  | 00:02:34.554 INFO    rq.worker :
nautobot-docker-nautobot-worker-1  |   Cleaning registries for queue: webhooks
nautobot-docker-nautobot-1         |   Applying dcim.0001_initial_part_1... OK
nautobot-docker-nautobot-1         |   Applying dcim.0002_initial_part_2... OK
nautobot-docker-nautobot-1         |   Applying ipam.0001_initial_part_1... OK
nautobot-docker-celery_beat-1      | __    -    ... __   -        _
nautobot-docker-celery_beat-1      | LocalTime -> 2022-09-05 00:02:36
nautobot-docker-celery_beat-1      | Configuration ->
nautobot-docker-celery_beat-1      |     . broker -> redis://:**@redis:6379/0
nautobot-docker-celery_beat-1      |     . loader -> celery.loaders.app.AppLoader
nautobot-docker-celery_beat-1      |     . scheduler -> nautobot.core.celery.schedulers.NautobotDatabaseScheduler
nautobot-docker-celery_beat-1      |
nautobot-docker-celery_beat-1      |     . logfile -> [stderr]@%WARNING
nautobot-docker-celery_beat-1      |     . maxinterval -> 5.00 seconds (5s)
nautobot-docker-postgres-1         | 2022-09-05 00:02:36.356 EEST [86] ERROR:  relation "extras_scheduledjob" does not exist at character 746
nautobot-docker-postgres-1         | 2022-09-05 00:02:36.356 EEST [86] STATEMENT:  SELECT "extras_scheduledjob"."id", "extras_scheduledjob"."name", "extras_scheduledjob"."task", "extras_scheduledjob"."job_model_id", "extras_scheduledjob"."job_class", "extras_scheduledjob"."interval", "extras_scheduledjob"."args", "extras_scheduledjob"."kwargs", "extras_scheduledjob"."queue", "extras_scheduledjob"."one_off", "extras_scheduledjob"."start_time", "extras_scheduledjob"."enabled", "extras_scheduledjob"."last_run_at", "extras_scheduledjob"."total_run_count", "extras_scheduledjob"."date_changed", "extras_scheduledjob"."description", "extras_scheduledjob"."user_id", "extras_scheduledjob"."approved_by_user_id", "extras_scheduledjob"."approval_required", "extras_scheduledjob"."approved_at", "extras_scheduledjob"."crontab" FROM "extras_scheduledjob" WHERE ("extras_scheduledjob"."enabled" AND (("extras_scheduledjob"."approval_required" AND "extras_scheduledjob"."approved_at" IS NOT NULL) OR NOT "extras_scheduledjob"."approval_required"))
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,356: CRITICAL/MainProcess] beat raised exception <class 'django.db.utils.ProgrammingError'>: ProgrammingError('relation "extras_scheduledjob" does not exist\nLINE 1: ...pproved_at", "extras_scheduledjob"."crontab" FROM "extras_sc...\n                                                             ^\n')
nautobot-docker-celery_beat-1      | Traceback (most recent call last):
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
nautobot-docker-celery_beat-1      |     return self.cursor.execute(sql, params)
nautobot-docker-celery_beat-1      | psycopg2.errors.UndefinedTable: relation "extras_scheduledjob" does not exist
nautobot-docker-celery_beat-1      | LINE 1: ...pproved_at", "extras_scheduledjob"."crontab" FROM "extras_sc...
nautobot-docker-celery_beat-1      |                                                              ^
nautobot-docker-celery_beat-1      |
nautobot-docker-celery_beat-1      |
nautobot-docker-celery_beat-1      | The above exception was the direct cause of the following exception:
nautobot-docker-celery_beat-1      |
nautobot-docker-celery_beat-1      | Traceback (most recent call last):
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/celery/apps/beat.py", line 105, in start_scheduler
nautobot-docker-celery_beat-1      |     service.start()
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/celery/beat.py", line 636, in start
nautobot-docker-celery_beat-1      |     humanize_seconds(self.scheduler.max_interval))
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/kombu/utils/objects.py", line 30, in __get__
nautobot-docker-celery_beat-1      |     return super().__get__(instance, owner)
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/functools.py", line 981, in __get__
nautobot-docker-celery_beat-1      |     val = self.func(instance)
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/celery/beat.py", line 679, in scheduler
nautobot-docker-celery_beat-1      |     return self.get_scheduler()
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/celery/beat.py", line 670, in get_scheduler
nautobot-docker-celery_beat-1      |     return symbol_by_name(self.scheduler_cls, aliases=aliases)(
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django_celery_beat/schedulers.py", line 231, in __init__
nautobot-docker-celery_beat-1      |     Scheduler.__init__(self, *args, **kwargs)
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/celery/beat.py", line 271, in __init__
nautobot-docker-celery_beat-1      |     self.setup_schedule()
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django_celery_beat/schedulers.py", line 239, in setup_schedule
nautobot-docker-celery_beat-1      |     self.install_default_entries(self.schedule)
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django_celery_beat/schedulers.py", line 361, in schedule
nautobot-docker-celery_beat-1      |     self._schedule = self.all_as_schedule()
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django_celery_beat/schedulers.py", line 245, in all_as_schedule
nautobot-docker-celery_beat-1      |     for model in self.Model.objects.enabled():
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 280, in __iter__
nautobot-docker-celery_beat-1      |     self._fetch_all()
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/cacheops/query.py", line 281, in _fetch_all
nautobot-docker-celery_beat-1      |     self._result_cache = list(self._iterable_class(self))
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 51, in __iter__
nautobot-docker-celery_beat-1      |     results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
nautobot-docker-celery_beat-1      |     cursor.execute(sql, params)
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/cacheops/transaction.py", line 97, in execute
nautobot-docker-celery_beat-1      |     result = self._no_monkey.execute(self, sql, params)
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 66, in execute
nautobot-docker-celery_beat-1      |     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
nautobot-docker-celery_beat-1      |     return executor(sql, params, many, context)
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in _execute
nautobot-docker-celery_beat-1      |     with self.db.wrap_database_errors:
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django/db/utils.py", line 90, in __exit__
nautobot-docker-celery_beat-1      |     raise dj_exc_value.with_traceback(traceback) from exc_value
nautobot-docker-celery_beat-1      |   File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
nautobot-docker-celery_beat-1      |     return self.cursor.execute(sql, params)
nautobot-docker-celery_beat-1      | django.db.utils.ProgrammingError: relation "extras_scheduledjob" does not exist
nautobot-docker-celery_beat-1      | LINE 1: ...pproved_at", "extras_scheduledjob"."crontab" FROM "extras_sc...
nautobot-docker-celery_beat-1      |                                                              ^
nautobot-docker-celery_beat-1      |
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,379: WARNING/MainProcess] Traceback (most recent call last):
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,379: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,380: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,381: WARNING/MainProcess] return self.cursor.execute(sql, params)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,381: WARNING/MainProcess] psycopg2.errors
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,381: WARNING/MainProcess] .
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,387: WARNING/MainProcess] UndefinedTable
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,387: WARNING/MainProcess] :
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,387: WARNING/MainProcess] relation "extras_scheduledjob" does not exist
nautobot-docker-celery_beat-1      | LINE 1: ...pproved_at", "extras_scheduledjob"."crontab" FROM "extras_sc...
nautobot-docker-celery_beat-1      |                                                              ^
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,387: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | The above exception was the direct cause of the following exception:
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,388: WARNING/MainProcess] Traceback (most recent call last):
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,388: WARNING/MainProcess]   File "/usr/local/bin/nautobot-server", line 8, in <module>
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,389: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,389: WARNING/MainProcess] sys.exit(main())
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,390: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/nautobot/core/cli.py", line 54, in main
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,391: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,391: WARNING/MainProcess] run_app(
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,391: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/nautobot/core/runner/runner.py", line 266, in run_app
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,392: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,392: WARNING/MainProcess] management.execute_from_command_line([runner_name, command] + command_args)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,393: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,394: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,394: WARNING/MainProcess] utility.execute()
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,395: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 413, in execute
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,396: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,396: WARNING/MainProcess] self.fetch_command(subcommand).run_from_argv(self.argv)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,396: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/nautobot/core/management/commands/celery.py", line 24, in run_from_argv
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,396: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,397: WARNING/MainProcess] celery_main.main()
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,397: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1055, in main
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,398: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,398: WARNING/MainProcess] rv = self.invoke(ctx)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,398: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,399: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,410: WARNING/MainProcess] return _process_result(sub_ctx.command.invoke(sub_ctx))
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,411: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,412: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,412: WARNING/MainProcess] return ctx.invoke(self.callback, **ctx.params)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,412: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/click/core.py", line 760, in invoke
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,413: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,413: WARNING/MainProcess] return __callback(*args, **kwargs)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,413: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,414: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,414: WARNING/MainProcess] return f(get_current_context(), *args, **kwargs)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,414: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/celery/bin/base.py", line 134, in caller
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,415: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,415: WARNING/MainProcess] return f(ctx, *args, **kwargs)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,416: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/celery/bin/beat.py", line 72, in beat
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,416: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,417: WARNING/MainProcess] return beat().run()
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,417: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/celery/apps/beat.py", line 77, in run
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,417: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,417: WARNING/MainProcess] self.start_scheduler()
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,418: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/celery/apps/beat.py", line 105, in start_scheduler
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,418: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,419: WARNING/MainProcess] service.start()
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,419: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/celery/beat.py", line 636, in start
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,423: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,423: WARNING/MainProcess] humanize_seconds(self.scheduler.max_interval))
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,423: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/kombu/utils/objects.py", line 30, in __get__
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,424: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,424: WARNING/MainProcess] return super().__get__(instance, owner)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,424: WARNING/MainProcess]   File "/usr/local/lib/python3.10/functools.py", line 981, in __get__
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,425: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,425: WARNING/MainProcess] val = self.func(instance)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,425: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/celery/beat.py", line 679, in scheduler
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,426: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,426: WARNING/MainProcess] return self.get_scheduler()
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,426: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/celery/beat.py", line 670, in get_scheduler
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,442: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,443: WARNING/MainProcess] return symbol_by_name(self.scheduler_cls, aliases=aliases)(
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,443: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django_celery_beat/schedulers.py", line 231, in __init__
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,443: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,444: WARNING/MainProcess] Scheduler.__init__(self, *args, **kwargs)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,444: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/celery/beat.py", line 271, in __init__
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,444: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,444: WARNING/MainProcess] self.setup_schedule()
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,444: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django_celery_beat/schedulers.py", line 239, in setup_schedule
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,446: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,446: WARNING/MainProcess] self.install_default_entries(self.schedule)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,447: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django_celery_beat/schedulers.py", line 361, in schedule
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,447: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,447: WARNING/MainProcess] self._schedule = self.all_as_schedule()
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,448: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django_celery_beat/schedulers.py", line 245, in all_as_schedule
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,448: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,449: WARNING/MainProcess] for model in self.Model.objects.enabled():
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,449: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 280, in __iter__
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,450: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,450: WARNING/MainProcess] self._fetch_all()
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,450: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/cacheops/query.py", line 281, in _fetch_all
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,451: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,451: WARNING/MainProcess] self._result_cache = list(self._iterable_class(self))
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,451: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django/db/models/query.py", line 51, in __iter__
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,452: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,452: WARNING/MainProcess] results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,452: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,453: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,453: WARNING/MainProcess] cursor.execute(sql, params)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,454: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/cacheops/transaction.py", line 97, in execute
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,454: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,454: WARNING/MainProcess] result = self._no_monkey.execute(self, sql, params)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,455: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 66, in execute
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,455: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,456: WARNING/MainProcess] return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,456: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,456: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,456: WARNING/MainProcess] return executor(sql, params, many, context)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,457: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in _execute
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,457: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,467: WARNING/MainProcess] with self.db.wrap_database_errors:
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,467: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django/db/utils.py", line 90, in __exit__
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,468: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,468: WARNING/MainProcess] raise dj_exc_value.with_traceback(traceback) from exc_value
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,468: WARNING/MainProcess]   File "/usr/local/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,469: WARNING/MainProcess]
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,469: WARNING/MainProcess] return self.cursor.execute(sql, params)
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,469: WARNING/MainProcess] django.db.utils
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,470: WARNING/MainProcess] .
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,470: WARNING/MainProcess] ProgrammingError
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,470: WARNING/MainProcess] :
nautobot-docker-celery_beat-1      | [2022-09-05 00:02:36,471: WARNING/MainProcess] relation "extras_scheduledjob" does not exist
nautobot-docker-celery_beat-1      | LINE 1: ...pproved_at", "extras_scheduledjob"."crontab" FROM "extras_sc...
nautobot-docker-celery_beat-1      |                                                              ^
nautobot-docker-nautobot-1         |   Applying extras.0002_initial_part_2... OK
nautobot-docker-celery_worker-1    |
nautobot-docker-celery_worker-1    |  -------------- celery@7f9b141e6f50 v5.2.7 (dawn-chorus)
nautobot-docker-celery_worker-1    | --- ***** -----
nautobot-docker-celery_worker-1    | -- ******* ---- Linux-5.15.0-47-generic-x86_64-with-glibc2.31 2022-09-05 00:02:37
nautobot-docker-celery_worker-1    | - *** --- * ---
nautobot-docker-celery_worker-1    | - ** ---------- [config]
nautobot-docker-celery_worker-1    | - ** ---------- .> app:         nautobot:0x7f4def263f10
nautobot-docker-celery_worker-1    | - ** ---------- .> transport:   redis://:**@redis:6379/0
nautobot-docker-celery_worker-1    | - ** ---------- .> results:     redis://:**@redis:6379/0
nautobot-docker-celery_worker-1    | - *** --- * --- .> concurrency: 2 (prefork)
nautobot-docker-celery_worker-1    | -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
nautobot-docker-celery_worker-1    | --- ***** -----
nautobot-docker-celery_worker-1    |  -------------- [queues]
nautobot-docker-celery_worker-1    |                 .> celery           exchange=celery(direct) key=celery
nautobot-docker-celery_worker-1    |
nautobot-docker-celery_worker-1    |
nautobot-docker-celery_beat-1 exited with code 1

None of those are repeated if the db exists already when creating the containers again (e.g. if we don't delete volumes between tests).

jdrew82 commented 2 months ago

This is a known issue with Celery or Beats starting before the database schema has been fully created. You can safely ignore it and fix it by allowing the nautobot container to finish the migrations and then restart all the containers. This should hopefully be less of an issue with some of the healthchecks that are in the compose files now.