Monthly tables are missing, leading to server errors

[Florian-Kranz]

Summary:

Yesterday everything was working fine, today quizzes are not working. Log says:

[CANVAS_ERRORS] EXCEPTION LOG ActiveRecord::StatementInvalid (PG::UndefinedTable: ERROR: relation "public.quiz_submission_events_2024_3" does not exist LINE 1: INSERT INTO "public"."quiz_submission_events_2024_3" ("attem...

I found #1945 but running Messages::Partitioner.process to create the missing tables returns => [].

Digging deeper, i found out, that actually none of the 'monthly tables' have been created. For example auditor_feature_flag_records_2024_3 and auditor_grade_change_records_2024_3 are missing too.

Steps to reproduce:

Not sure.

Expected behavior:

Monthly tables should be created automatically. (Worked up to this point)

Actual behavior:

Monthly tables were not created automatically.

Additional notes:

The instance has always been stable. No changes to the configuration has been made recently.

[amg-web]

i would check if canvas_init is running

[Florian-Kranz]

Thanks @amg-web for the reply!

Today canvas is working again, i assume tonight a scheduled job ran and created the tables.

Maybe the reaseon for not creating could have been some sort of timezone missconfiguration and the fact that its a leap year?? I really dont know, just poking in the dark...

[amg-web]

there is only one reason: canvas_init is down more then 2 weeks

[Florian-Kranz]

Alright, thanks for the clarification. For future reference, how do i make sure it is running?

[amg-web]

you can try to schedule restart, e.g. weekly