🐛 Good Jobs should process jobs

[ShanaLMoore]

Summary

When I manually create a work via the UI and attach a file, I don't see jobs running in good_job. The work never attaches the file I selected.

Acceptance Criteria

• [ ] Good jobs should run background jobs

Testing Instructions

• create a manual work. attach a file to it.
• visit /jobs endpoint to confirm the counts are zero in the scheduled and running queues (meaning that the jobs have finished processing). It should take a few minutes.
• refresh the work's show page to confirm that the file got attached.

Screenshots

[ShanaLMoore]

good_job doesn't appear to queue anything up when creating a work

tested via bulkrax too

Sample file:

Test of URLs in CSV upload.csv

This should not be zero

[ShanaLMoore]

When initiating the index rake task:

But GoodJob::Job.count => 0

When called directly:

ReindexWorksJob.perform_later

GoodJob::Job.count => 1

[ShanaLMoore]

disregard some of the above comments, I may have forgotten to switch tenants in the console. Trying again:

bundle exec rails index_works => Enqueued ReindexWorksJob (Job ID: 3277628f-9c3b-406f-9fd8-fbdab2540025) to GoodJob(default)

GoodJob::Job.count => 1

GoodJob::Job.find_by(active_job_id: "3277628f-9c3b-406f-9fd8-fbdab2540025") => #<GoodJob::Job id: "737a71c4-cd61-47f8-9527-2f4e9c91c4c3", queue_name: "default", priority: 0, serialized_params: {"job_id"=>"3277628f-9c3b-406f-9fd8-fbdab2540025", "locale"=>"en", "tenant"=>"5f5840c0-f5c1-41d1-bd0f-ecc2b400f62d", "priority"=>0, "arguments"=>[], "job_class"=>"ReindexWorksJob", "executions"=>0, "queue_name"=>"default", "provider_job_id"=>nil}, scheduled_at: nil, performed_at: nil, finished_at: nil, error: nil, created_at: "2023-09-27 18:04:45", updated_at: "2023-09-27 18:04:45", active_job_id: "3277628f-9c3b-406f-9fd8-fbdab2540025", concurrency_key: nil, cron_key: nil, retried_good_job_id: nil, cron_at: nil>

But when visiting the dashboard all the values still say 0

It appears reindex job is failing on NoMethodError: undefined method `available_works'. Perhaps the other jobs are failing for the same reason and not reporting it to the dashboard so I'll debug this area.

This is erroring because Account.global_tenant? returns true. In rancher/staging it returns false. DIff:

  def self.global_tenant?
    # Global tenant only exists when multitenancy is enabled and NOT in test environment
    # (In test environment tenant switching is currently not possible)
    return false unless ActiveModel::Type::Boolean.new.cast(ENV.fetch('HYKU_MULTITENANT', false)) && !Rails.env.test?
    Apartment::Tenant.default_tenant == Apartment::Tenant.current
  end

Apartment::Tenant.default_tenant == Apartment::Tenant.current => true # locally which is the diff.

Locally Apartment::Tenant.current => 'public'. it seems like there needs to be a switch happening somewhere, before it gets to this point

[ShanaLMoore]

cleared all the jobs to re orient myself. I kicked off a bulkrax importer and can see this in the terminal but nothing in the dashboard still. I noticed that following job has a que name "import" but I don't see that option in the dashboard drop down

GoodJob::Job.all.pluck(:serialized_params).map {|k,v| k['jo b_class']} (0.7ms) SELECT "good_jobs"."serialized_params" FROM "good_jobs" => ["ReindexWorksJob", "ReindexCollectionsJob", "ReindexWorksJob", "ReindexWorksJob", "Bulkrax::ImporterJob", "Bulkrax::ImporterJob", "ContentUpdateEventJob", "AttachFilesToWorkJob"]

#<GoodJob::Job id: "184a77a2-b5de-41bf-a731-6a4243c04c69", queue_name: "import", priority: -20, serialized_params: {"job_id"=>"be969512-855e-4759-b1bc-a67603656b43", "locale"=>"en", "tenant"=>"5f5840c0-f5c1-41d1-bd0f-ecc2b400f62d", "priority"=>-20, "arguments"=>[2, false], "job_class"=>"Bulkrax::ImporterJob", "executions"=>0, "queue_name"=>"import", "provider_job_id"=>nil}, scheduled_at: nil, performed_at: nil, finished_at: nil, error: nil, created_at: "2023-09-27 19:41:00", updated_at: "2023-09-27 19:41:00", active_job_id: "be969512-855e-4759-b1bc-a67603656b43", concurrency_key: nil, cron_key: nil, retried_good_job_id: nil, cron_at: nil>

GoodJob::Job.all.map {|j| j.status} => [:queued, :queued, :queued, :queued, :queued, :queued, :queued, :queued]

[ShanaLMoore]

Switched env var to sidekiq just to see if it'll work.

It immediately queued in the dashboard as expected. I attached a file to a work via the UI.

But an error occurred:

After correcting the above error I was able to get the file to attach, but see another error with create derv jobs.

[ShanaLMoore]

I haven't been able to figure this out yet so I'm going to move it back and pick up another ticket for the sake of progress.

If others need to save a file for testing purposes, use sidekiq for now.

I switched to running hyku proper and their /jobs endpoint loads sidekiq so leaving that for now.

[ShanaLMoore]

Reading through the good_jobs doc, this seems to describe the behavior I'm seeing

:external causes the adapter to enqueue jobs, but not execute them. When using this option (the default for production environments), you'll need to use the command-line tool to actually execute your jobs.

I changed external to inline. It successfully attached the work but uses the rails server instead so I'll keep trying other configurations and/or try to figure out why external works for adventist locally but not for this project.

[laritakr]

@ShanaLMoore is this one done now?

[jillpe]

SoftServ QA: ✅ Pass!

• Was able see the jobs running and scheduled when creating a work
• Watched the jobs complete
• File was attached to work's show page

Screenshots