Error message in production console

[MarmosetMusic]

I recently started using the gem, but every time I open my production console (Heroku CLI) this message comes up, twice. Any ideas?

Connecting to database specified by DATABASE_URL

Expected(200) <=> Actual(401 Unauthorized) request => {:connect_timeout=>60, :headers=>{"Accept"=>"application/json", "Accept-Encoding"=>"gzip", "Authorization"=>"Basic OmQ5NDc0NTY0LWUwMDMtNDRlYy1hYTJjLTUxODYwMTI5YzA5Zg==", "User-Agent"=>"heroku-rb/0.2.1", "X-Heroku-API-Version"=>"3", "X-Ruby-Version"=>"2.0.0", "X-Ruby-Platform"=>"x86_64-linux", "Host"=>"api.heroku.com:443"}, :instrumentor_name=>"excon", :mock=>false, :read_timeout=>60, :retry_limit=>4, :ssl_ca_file=>"/app/vendor/bundle/ruby/2.0.0/gems/excon-0.13.4/data/cacert.pem", :ssl_verify_peer=>true, :write_timeout=>60, :host=>"api.heroku.com", :path=>"/apps/staging-xxxx/ps", :port=>"443", :query=>nil, :scheme=>"https", :expects=>200, :method=>:get} response => #<Excon::Response:0x007f2ecf385658 @body="{\"id\":\"unauthorized\",\"error\":\"Invalid credentials provided.\"}", @headers={"Cache-Control"=>"no-transform", "Content-Type"=>"application/json", "Date"=>"Tue, 28 Apr 2015 18:40:19 GMT", "Request-Id"=>"53c8af5d-0d1c-46bf-8779-e5f6d6c8cd1a", "Server"=>"nginx/1.4.6 (Ubuntu)", "Status"=>"401 Unauthorized", "X-Content-Type-Options"=>"nosniff", "X-Runtime"=>"0.015607713", "Content-Length"=>"61", "Connection"=>"keep-alive"}, @status=401> Expected(200) <=> Actual(401 Unauthorized)

[afn]

Did you end up figuring this out? It doesn't seem to be related to delayed_job_recurring, unless I'm missing something.

[MarmosetMusic]

I removed the gem from my staging app and this message went away. I am using it in production (and is working well!), but, the above message is still there.

If it is helpful, I have this in an initializer: RemoveArchivedTrack.schedule! if Rails.env.production? and created the job in my lib/ folder, in it is a bunch of email related code, but at the top, I have this:

include Delayed::RecurringJob run_every 1.week run_at 'sunday 1:00am' timezone 'US/Pacific' queue 'slow-jobs'

[afn]

Are you using the workless gem? It looks like it's unable to connect to the Heroku API to spin up a worker. Take a look at this SO answer: http://stackoverflow.com/questions/13367838/401-unauthorized-errors-on-heroku-with-delayed-job. I believe you need to set the HEROKU_API_KEY and APP_NAME configuration variables as described here.

[MarmosetMusic]

Yes, I am using workless. The HEROKU_API_KEY and APP_NAME config vars are set.

[afn]

If you remove your initializer and manually call RemoveArchivedTrack.schedule! instead, do you still get the error?

[MarmosetMusic]

I removed my initializer. Good news, that error doesn't pop up every time I open the heroku console. But when I manually call RemoveArchivedTrack.schedule!, I get the same error. I deleted the actual scheduled job from the db, ran that command, and it successfully created the scheduled job, even though that same 401 error came up.

[afn]

Do you get the same error when you schedule a delayed job manually? E.g. try running something like:

delay.puts "this is a test"

Does the job get executed? Or do you get the same error as you do with the recurring job?

[MarmosetMusic]

This is what I get when i run:

delay.puts "this is a test"

(1.9ms) BEGIN SQL (7.6ms) INSERT INTO "delayed_jobs" ("attempts", "created_at", "failed_at", "handler", "last_error", "locked_at", "locked_by", "priority", "queue", "run_at", "updated_at") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) RETURNING "id" [["attempts", 0], ["created_at", Tue, 26 May 2015 13:39:03 PDT -07:00], ["failed_at", nil], ["handler", "--- !ruby/object:Delayed::PerformableMethod\nobject: !ruby/object {}\nmethod_name: :puts\nargs:\n- this is a test\n"], ["last_error", nil], ["locked_at", nil], ["locked_by", nil], ["priority", 0], ["queue", nil], ["run_at", Tue, 26 May 2015 13:39:03 PDT -07:00], ["updated_at", Tue, 26 May 2015 13:39:03 PDT -07:00]] (5.6ms) COMMIT

#<Heroku::API::Errors::Unauthorized: Expected(200) <=> Actual(401 Unauthorized) request => {:connect_timeout=>60, :headers=>{"Accept"=>"application/json", "Accept-Encoding"=>"gzip", "Authorization"=>"Basic OmQ5NDc0NTY0LWUwMDMtNDRlYy1hYTJjLTUxODYwMTI5YzA5Zg==", "User-Agent"=>"heroku-rb/0.2.1", "X-Heroku-API-Version"=>"3", "X-Ruby-Version"=>"2.0.0", "X-Ruby-Platform"=>"x86_64-linux", "Host"=>"api.heroku.com:443"}, :instrumentor_name=>"excon", :mock=>false, :read_timeout=>60, :retry_limit=>4, :ssl_ca_file=>"/app/vendor/bundle/ruby/2.0.0/gems/excon-0.13.4/data/cacert.pem", :ssl_verify_peer=>true, :write_timeout=>60, :host=>"api.heroku.com", :path=>"/apps/marmosetmusic/ps", :port=>"443", :query=>nil, :scheme=>"https", :expects=>200, :method=>:get} response => #<Excon::Response:0x007f9e536a3788 @body="{\"id\":\"unauthorized\",\"error\":\"Invalid credentials provided.\"}", @headers={"Cache-Control"=>"no-transform", "Content-Type"=>"application/json", "Date"=>"Tue, 26 May 2015 20:39:03 GMT", "Request-Id"=>"646ec9cd-f896-494a-baf0-85803a9c6d96", "Server"=>"nginx/1.4.6 (Ubuntu)", "Status"=>"401 Unauthorized", "X-Content-Type-Options"=>"nosniff", "X-Runtime"=>"0.014968626", "Content-Length"=>"61", "Connection"=>"keep-alive"}, @status=401>> => #<Delayed::Backend::ActiveRecord::Job id: 115847, priority: 0, attempts: 0, handler: "--- !ruby/object:Delayed::PerformableMethod\nobject:...", last_error: nil, run_at: "2015-05-26 20:39:03", locked_at: nil, failed_at: nil, locked_by: nil, queue: nil, created_at: "2015-05-26 20:39:03", updated_at: "2015-05-26 20:39:03">

[MarmosetMusic]

Thanks for all your help, @afn. I figured it out. While the error went away when I removed delayed_job_recurring, I believe was an issue with workless. I had changed my heroku password, which I didn't know automatically generates a new api key. I used heroku config:set -a app-name HEROKU_API_KEY=new_api_key and the message is gone. I read this comment on Stack Overflow which pointed me on the right direction:

"I had this error after changing my account password via Heroku's website, which changed the API key, which meant that until I updated HEROKU_API_KEY, workless complained with an unauthorized error similar to the one you sighted"

Again, thanks for the help @afn. This issue can be closed.