remove Active Record dependency

[westonplatter]

I'm on MongoDB / Mongoid and would like to pull this in. I see you're using AcitveRecord for ORM associations and transctions, and ActiveImport for mass insertion.

Is it insane of me to think of me moving that functionality to a Ruby class mapped to the Redis store?

Any work currently going to move remvoe the AR dependency?

[tombenner]

Moving this to a Redis store is fairly feasible; there's only one model (Subjob), and the lookups are pretty simple and homogenous. The implementation might just consist of a new Subjob class (as you mention) and replacing some the AR methods called on it elsewhere in the code. I'm sure other users would appreciate the improved performance and removal of dependencies, too.

We (SocialPandas) rely on Subjob records being in Postgres, so we'll need to preserve support for AR. If you're able to allow users to choose between Redis or AR, that'd be ideal, but if you'd rather just create a Redis-only fork, we'd very happily link to your fork in our README.

I haven't heard of anyone else working on removing the AR dependency.

[westonplatter]

@tombenner thanks for the response.

I think it would make the most sense to fork the repo and focus first on a redis storage and then come back for a configurable store option (similar to Devise).

[tombenner]

Sounds good!

[lucasgertel]

It's possible to add use it with mongoID?

[DarthMax]

Since it seems that nothing has happened yet I started a port that uses redis instead of active record. I tried to change as little as possible of the actual code basis and only port the subjob.rb class to use redis instead of AR make it possible to merge both together in the future.

Nevertheless since one can only query by the key in redis it was necessary to change the way subjob_id, id and jid of the subjobs are generated and used (basically they are all the same now). They now use the format 'sj:SUPERJOBID/SUBJOBID'

In the original code the children_ids are not set for workers of type batchjob_child which is necessary as I cant find the children by querying for the parent_id.

Unfortunately I have no idea how one could integrate this into the actual superworker project.

The code can be found here https://github.com/DarthMax/sidekiq-superworker/tree/redis-backend