SqsActiveJob doesn't need Rails

[ckhsponge]

Thanks for creating SqsActiveJob! I've tested it and it seems to work well.

My feature request is to split SqsActiveJob into it's own gem and remove the dependency on Rails. The ActiveJob gem does not require Rails. Lambdas can run Ruby. A lightweight Lambda could have a few gems not including Rails and be very effective at background job processing.

I've tested the project after removing 3 Rails references:

1. Logger - A Rails logger is used. To fix this, check if Rails exists. If not, then use the standard Logger instead of the Rails logger.
2. Config - A configuration file existence is checked in the Rails root. Don't look for this file if Rails doesn't exist.
3. Polling - The Rails environment is loaded before polling. Don't do this. Instead start whatever environment is appropriate and then initiate polling.

I'd be happy to split this project out myself however not maintaining it in 2 places would be better.

[mullermp]

Interesting. I had a dream (or nightmare) last night about splitting out features of aws-sdk-rails into separate gems. I originally made that decision some years ago to drive usage and make maintenance easier but it's actually been the opposite. I think it would be backwards compatible to pull it out, have the lastest aws-sdk-rails depend on it, and maintain multiple gems in the aws-sdk-rails repo. If you'd like to share that work, we can track it here, but no guarantees on timeline. @alextwoods @jterapin thoughts?

[ckhsponge]

I like that I idea. I can try to get a PR put together.

[ckhsponge]

@mullermp Here's a PR. Moving the directories around and creating the gemspsec was straightforward. Only minor changes were needed to remove Rails references. Any thoughts?

https://github.com/aws/aws-sdk-rails/pull/144

[januszm]

Could we also add a clear description for how to set the route "/" or different path for the Middleware to accept Cron tasks? I'm not sure if by default it'll just work with "/", I noticed that the "/" is used by the action_mailbox. I mean for cron.yaml url: setting.

[mullermp]

I'm working on splitting out gems, starting with some of the easy ones (sessionstore and aws-record). I'll use your PR for inspiration when I arrive at ActiveJob. We have another story to solve related to this, related to automated releasing, so it may take some time to solve that. We are debating on mono repo versus multi repo for all of these feature gems.

With regards to this gem, the current plan is to make a new minor version release with defined?() checks on various features while continuing to depend on the gems in the gemspec, then a new major version where you need to add the feature gems to your Gemfile.

[ckhsponge]

Thanks for working on this. We've been using my branch with the Rails references removed in production and it works fine.

For context, we have a Sinatra app and the job handling is done with:

# load models and jobs
require_relative 'application'

class TaskHandler
  def self.job(event:, context:)
    Aws::Rails::SqsActiveJob.lambda_job_handler(event: event, context: context)
  end
end

[admtnnr]

I'm working on splitting out gems, starting with some of the easy ones (sessionstore and aws-record). I'll use your PR for inspiration when I arrive at ActiveJob. We have another story to solve related to this, related to automated releasing, so it may take some time to solve that. We are debating on mono repo versus multi repo for all of these feature gems.

@mullermp Not to hijack, but just wanted to show support and say this is great to hear. I was just debating whether to extract just the ActionMailer SESv2 functionality out into it's own gem or embed it in our application directly. I want the native integration so that I can rely on the instance role for permissions instead of generating an IAM user for SMTP for compliance reasons (constantly rotating access keys is a PITA), but don't want/need all of the other functionality provided.

[mullermp]

@admtnnr I have a PR that refactors/refreshes the Action Mailers but I kept it in the gem. Are you using Action Mailer without rails?

[mullermp]

@ckhsponge Can you tell me if you are using the queue adapters without rails? My intuition says no. In your PR you moved those adapters into the generic gem, but I think they might still belong in aws-sdk-rails. Can you show me usage on how you're using this feature generically?

[ckhsponge]

@mullermp The name of the repo is still aws-sdk-rails but I'm only loading the sqs-active-job gem from it. I also am using the activejob gem from the rails repo but not actually loading Rails.

gem "activejob", "~> 7.2"
gem "sqs-active-job", "~> 4.0", github: 'ckhsponge/aws-sdk-rails', branch: 'sqs-active-job'

Here's my initializer:

require 'active_job'
require 'active_job/queue_adapters'
require 'sqs_active_job'

ActiveJob::Base.queue_adapter = (ENV.fetch('ACTIVE_JOB_QUEUE_ADAPTER') { 'sqs' }).to_sym

if ENV['JOB_AWS_SQS_URL'] && ENV.fetch('ACTIVE_JOB_QUEUE_ADAPTER') { 'sqs' } == 'sqs'
  Aws::Rails::SqsActiveJob.configure do |config|
    config.queues = { default: ENV['JOB_AWS_SQS_URL'] }
  end
end

For developing I set ACTIVE_JOB_QUEUE_ADAPTER=async although I have run with sqs locally using localstack.

Looks like I moved some /lib files including queue adapters into /sqs-active-job/lib which I think is needed if we're encapsulating into that directory.

[mullermp]

Thanks. It seems like the answer is yes, you still need the queue adapters, because that's how you set them on ActiveJob::Base. I finally cut through the red tape of Amazon and got a few github repos created - I will be posting a PR soon to https://github.com/aws/aws-activejob-sqs-ruby. I plan to use a different namespace and map it in aws-sdk-rails in the current major version. If wanted to collaborate or test what I have, that would be great, just give me some time.

[ckhsponge]

Yes, indeed.

I'd be happy to collaborate and definitely could test.

[admtnnr]

@admtnnr I have a PR that refactors/refreshes the Action Mailers but I kept it in the gem. Are you using Action Mailer without rails?

I am not. It will still be in the context of a Rails application, I just didn't need the rest of the AWS Rails SDK functionality. It looks like you've already started the extraction though in https://github.com/aws/aws-actionmailer-ses-ruby which is awesome and greatly appreciated! Also happy to collaborate and test as needed.

[mullermp]

@admtnnr If you can pull that gem from github branch init and give it a test it would be greatly appreciated. I changed the mailer names, so you'll have to pull https://github.com/aws/aws-sdk-rails/pull/159/files for backwards compatibility or make the small change.