bobf / action_mailbox_amazon_ingress

Rails ActionMailbox ingress for Amazon SES
MIT License
28 stars 17 forks source link

ActionMailbox::Router::RoutingError (ActionMailbox::Router::RoutingError) #1

Closed romarioliveira25 closed 4 years ago

romarioliveira25 commented 4 years ago

First of all, thanks for your contribution with this gem, was a greate initiave after discontinuing native support.

So, after make all setup on AWS SES (Verifying domain and Email receive rule sets) and SNS (Creating topics and related subscriptions), I have experieced the following error.

The POST request is fired from AWS and the tables action_mailbox_inbound_emails, active_storage_blobs, active_storage_attachments are filled, but on performing ActionMailbox::RoutingJob the error occurs.

Can provide any help to solve this?

Error performing ActionMailbox::RoutingJob (Job ID: 25ce2a46-7fb7-4085-a492-27ff3b6be3d1) from Sidekiq(action_mailbox_routing) in 10.53ms: ActionMailbox::Router::RoutingError (ActionMailbox::Router::RoutingError):
/usr/local/bundle/ruby/2.6.0/gems/actionmailbox-6.0.2.1/lib/action_mailbox/router.rb:29:in `route'
/usr/local/bundle/ruby/2.6.0/gems/actionmailbox-6.0.2.1/lib/action_mailbox/routing.rb:18:in `route'
/usr/local/bundle/ruby/2.6.0/gems/actionmailbox-6.0.2.1/app/models/action_mailbox/inbound_email/routable.rb:22:in `route'
/usr/local/bundle/ruby/2.6.0/gems/actionmailbox-6.0.2.1/app/jobs/action_mailbox/routing_job.rb:10:in `perform'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/execution.rb:39:in `block in perform_now'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
/usr/local/bundle/ruby/2.6.0/gems/sentry-raven-2.13.0/lib/raven/integrations/rails/active_job.rb:13:in `block (2 levels) in included'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:121:in `instance_exec'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/usr/local/bundle/ruby/2.6.0/gems/i18n-1.7.1/lib/i18n.rb:313:in `with_locale'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/translation.rb:9:in `block (2 levels) in <module:Translation>'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:121:in `instance_exec'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/core_ext/time/zones.rb:66:in `use_zone'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/timezones.rb:9:in `block (2 levels) in <module:Timezones>'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:121:in `instance_exec'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/logging.rb:25:in `block (4 levels) in <module:Logging>'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/notifications.rb:180:in `block in instrument'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/notifications.rb:180:in `instrument'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/logging.rb:24:in `block (3 levels) in <module:Logging>'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/logging.rb:45:in `block in tag_logger'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/tagged_logging.rb:80:in `block in tagged'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/tagged_logging.rb:28:in `tagged'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/tagged_logging.rb:80:in `tagged'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/logging.rb:45:in `tag_logger'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/logging.rb:21:in `block (2 levels) in <module:Logging>'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:121:in `instance_exec'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:139:in `run_callbacks'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/execution.rb:38:in `perform_now'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/execution.rb:24:in `block in execute'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:112:in `block in run_callbacks'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/railtie.rb:43:in `block (4 levels) in <class:Railtie>'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/execution_wrapper.rb:88:in `wrap'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/reloader.rb:72:in `block in wrap'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/execution_wrapper.rb:84:in `wrap'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/reloader.rb:71:in `wrap'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/railtie.rb:42:in `block (3 levels) in <class:Railtie>'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:121:in `instance_exec'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/callbacks.rb:139:in `run_callbacks'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/execution.rb:22:in `execute'
/usr/local/bundle/ruby/2.6.0/gems/activejob-6.0.2.1/lib/active_job/queue_adapters/sidekiq_adapter.rb:42:in `perform'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:196:in `execute_job'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/middleware/chain.rb:138:in `block in invoke'
/usr/local/bundle/ruby/2.6.0/gems/apartment-sidekiq-1.2.0/lib/apartment/sidekiq/middleware/server.rb:5:in `block in call'
/usr/local/bundle/ruby/2.6.0/bundler/gems/apartment-f266f73e5883/lib/apartment/adapters/abstract_adapter.rb:85:in `switch'
/usr/local/lib/ruby/2.6.0/forwardable.rb:230:in `switch'
/usr/local/bundle/ruby/2.6.0/gems/apartment-sidekiq-1.2.0/lib/apartment/sidekiq/middleware/server.rb:4:in `call'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/middleware/chain.rb:140:in `block in invoke'
/usr/local/bundle/ruby/2.6.0/gems/sentry-raven-2.13.0/lib/raven/integrations/sidekiq.rb:9:in `call'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/middleware/chain.rb:140:in `block in invoke'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/middleware/chain.rb:143:in `invoke'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:163:in `block in process'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/job_retry.rb:111:in `local'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/rails.rb:43:in `block in call'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/execution_wrapper.rb:88:in `wrap'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/reloader.rb:72:in `block in wrap'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/execution_wrapper.rb:88:in `wrap'
/usr/local/bundle/ruby/2.6.0/gems/activesupport-6.0.2.1/lib/active_support/reloader.rb:71:in `wrap'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/rails.rb:42:in `call'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:257:in `stats'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/job_logger.rb:13:in `call'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/job_retry.rb:78:in `global'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:124:in `block in dispatch'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/logger.rb:10:in `with'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/job_logger.rb:33:in `prepare'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:123:in `dispatch'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:162:in `process'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:78:in `process_one'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/processor.rb:68:in `run'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/util.rb:15:in `watchdog'
/usr/local/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/util.rb:24:in `block in safe_thread'
2020-07-21T22:24:28.056Z pid=1 tid=gqdwog9up class=ActionMailbox::RoutingJob jid=4b96377e50d4c752edaac288 elapsed=0.026 INFO: fail
2020-07-21T22:24:28.057Z pid=1 tid=gqdwog9up WARN: {"context":"Job raised exception","job":{"retry":true,"queue":"action_mailbox_routing","class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"ActionMailbox::RoutingJob","args":[{"job_class":"ActionMailbox::RoutingJob","job_id":"25ce2a46-7fb7-4085-a492-27ff3b6be3d1","provider_job_id":null,"queue_name":"action_mailbox_routing","priority":null,"arguments":[{"_aj_globalid":"gid://ping/ActionMailbox::InboundEmail/5"}],"executions":0,"exception_executions":{},"locale":"pt-BR","timezone":"Brasilia","enqueued_at":"2020-07-21T22:24:01Z"}],"jid":"4b96377e50d4c752edaac288","created_at":1595370241.1979206,"apartment":"public","enqueued_at":1595370268.0300646,"error_message":"ActionMailbox::Router::RoutingError","error_class":"ActionMailbox::Router::RoutingError","failed_at":1595370241.3972886,"retry_count":0},"jobstr":"{\"retry\":true,\"queue\":\"action_mailbox_routing\",\"class\":\"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper\",\"wrapped\":\"ActionMailbox::RoutingJob\",\"args\":[{\"job_class\":\"ActionMailbox::RoutingJob\",\"job_id\":\"25ce2a46-7fb7-4085-a492-27ff3b6be3d1\",\"provider_job_id\":null,\"queue_name\":\"action_mailbox_routing\",\"priority\":null,\"arguments\":[{\"_aj_globalid\":\"gid://ping/ActionMailbox::InboundEmail/5\"}],\"executions\":0,\"exception_executions\":{},\"locale\":\"pt-BR\",\"timezone\":\"Brasilia\",\"enqueued_at\":\"2020-07-21T22:24:01Z\"}],\"jid\":\"4b96377e50d4c752edaac288\",\"created_at\":1595370241.1979206,\"apartment\":\"public\",\"enqueued_at\":1595370268.0300646,\"error_message\":\"ActionMailbox::Router::RoutingError\",\"error_class\":\"ActionMailbox::Router::RoutingError\",\"failed_at\":1595370241.3972886,\"retry_count\":0}"}
bobf commented 4 years ago

@romarioliveira25 Hi, thanks for providing a detailed error report - I will try to take a closer look at this in my free time.

For the time being, you may be interested in this PR which (hopefully !) will mean SES integration will be included in Rails 6.1 - I am just waiting for the Rails devs to approve/provide further feedback on the PR: https://github.com/rails/rails/pull/39364

Will try to get back to you this weekend on your issues with the standalone gem.

romarioliveira25 commented 4 years ago

Perfect! Thank's for your time. And I found your gem at the rails PR haha

bobf commented 4 years ago

@romarioliveira25 I finally managed to find some time to look at this.

I was able to recreate the error:

[ActiveJob] [ActionMailbox::RoutingJob] [531cb2d2-d35d-484f-8d53-de7b2b937b4a] Error performing ActionMailbox::RoutingJob (Job ID: 531cb2d2-d35d-484f-8d53-de7b2b937b4a) from Async(action_mailbox_routing) in 39.07ms: ActionMailbox::Router::RoutingError (ActionMailbox::Router::RoutingError):

You just need to add some extra ActionMailbox configuration:

# app/mailboxes/application_mailbox.rb

class ApplicationMailbox < ActionMailbox::Base
  routing /.*/ => :defaults
end
# app/mailboxes/defaults_mailbox.rb
class DefaultsMailbox < ApplicationMailbox
  def process
    puts mail
  end
end

You'll need to adjust to suit your needs but you need to define at least one route so Rails/ActionMailbox will know where to route the incoming mail to.

I've created a demo repository for you here: https://github.com/bobf/action_mailbox_amazon_ingress_demo

You'll need to edit config/application.rb to set the ARN for the SNS topic but otherwise it should work out of the box - I tested using Ngrok and had no issues.

See this documentation for further reference: https://api.rubyonrails.org/classes/ActionMailbox/Base.html

Let me know how it goes !

romarioliveira25 commented 4 years ago

Hi again @bobf, one more time, thanks for your support.

I'm really wasn't create a default routing (although I'm not sure that issue is related with that)

About the SNS topic, I did the config as mentioned at the docs shared by you at the repo.

So, while I was waiting for your response, I have tested the griddler gem, but with that I checked one limitation to use AWS SES for receive email purposes related to the large payload, because I need receive emails with attachments and it exceeds the limit of 256KB in AWS SNS Notification.

AWS docs: https://docs.aws.amazon.com/sns/latest/dg/sns-large-payload-raw-message-delivery.html