Recurring tasks not being scheduled (no unique index found)

[tsvallender]

At some point my recurring tasks have stop being scheduled. I presume one of the upgrades has caused this but I’m unsure which—I didn’t immediately notice the tasks weren’t running.

This is my configuration:

default: &default
  dispatchers:
    - polling_interval: 1
      batch_size: 500
      concurrency_maintenance_interval: 300
      recurring_tasks:
        generate_journeys:
          class: GenerateJourneysJob
          schedule: every minute
        update_boat_designs:
          class: UpdateBoatDesignsJob
          schedule: every 15 minutes
        update_no_foreign_land:
          class: UpdateNoForeignLandJob
          schedule: every 15 minutes
  workers:
    - queues: "*"
      threads: 3
      processes: 1
      polling_interval: 0.1

development:
 <<: *default

production:
 <<: *default

I’m deploying with Kamal and run Solid Queue in its own container with bundle exec rake solid_queue:start.

I can see the jobs listed in mission_control-jobs:

But as you can see they’re no longer being enqueued. I also can’t see any errors being raised.

[tsvallender]

Tell a lie, just found the error:

I, [2024-08-22T10:41:00.025229 #14]  INFO -- : Enqueued GenerateJourneysJob (Job ID: 18bd37db-94ff-4f2c-870c-d584e7565b78) to SolidQueue(default)
D, [2024-08-22T10:41:00.025687 #17] DEBUG -- :   TRANSACTION (0.6ms)  BEGIN
D, [2024-08-22T10:41:00.026600 #17] DEBUG -- :   SolidQueue::ReadyExecution Exists? (1.4ms)  SELECT 1 AS one FROM "solid_queue_ready_executions" LIMIT $1 FOR UPDATE SKIP LOCKED  [["LIMIT", 1]]
D, [2024-08-22T10:41:00.027311 #17] DEBUG -- :   TRANSACTION (0.5ms)  COMMIT
D, [2024-08-22T10:41:00.027732 #14] DEBUG -- :   TRANSACTION (0.9ms)  ROLLBACK
E, [2024-08-22T10:41:00.028313 #14] ERROR -- : SolidQueue-0.6.0 Error enqueuing recurring task (19.7ms)  task: "generate_journeys", at: "2024-08-22T10:41:00Z"
E, [2024-08-22T10:41:00.028727 #14] ERROR -- : SolidQueue-0.6.0 Error in thread (0.0ms)  error: "ArgumentError No unique index found for [:task_key, :run_at]"

[rosa]

Hey @tsvallender, that's really strange because there's a unique index there from the very beginning 😕 Did you somehow create that table manually?

[tsvallender]

Hey @tsvallender, that's really strange because there's a unique index there from the very beginning 😕 Did you somehow create that table manually?

Hmm, no they were created with the migrations. In the morning I'll take a look and see what indexes there are. Strange indeed...

[tsvallender]

So looking at my schema I can see there is an index there, but without the unique constraint. Double checked and I do have unique set in the migration that created this table as normal, so I have no idea how this can have happened.

I added and ran a migration to recreate the index with a unique constraint and now things are working.

From my point of view this is solved, and I can’t see how this would occur for others, but I also can’t see how it ever occurred in the first place! That being said, I’ll close this issue but if there’s any investigation I can do that would be helpful I’m happy to, thanks.