malthe / pq

A PostgreSQL job queueing system
376 stars 41 forks source link

Any interest in porting to cockroach #54

Closed retr0h closed 4 years ago

retr0h commented 4 years ago

I was curious if you would be interested in porting to cockroach. As I understand it, cockroach has implemented SKIP LOCKED, and while I like pg, I am currently using cockroach b/c of it's wonderful clustering capabilities.

I do understand this request is probably out of scope for what this project is doing, but was curious if you might be interested if it is minimal effort.

John

stas commented 4 years ago

CockroachDB is built to be largely compatible with PostgreSQL, meaning that software written to use PostgreSQL can sometimes (often!) be used with CockroachDB without changes.

This suggests a lot of stuff should be working out of the box, would you be kind to give it a try and suggest what's not working?

retr0h commented 4 years ago

Yeah, I have started going down that path, and unfortunately it looks like most of the stuff we care about is unimplemented.

psycopg2.errors.FeatureNotSupported: unimplemented: SKIP LOCKED lock wait policy is not supported
HINT:  You have attempted to use a feature that is not yet implemented.
See: https://github.com/cockroachdb/cockroach/issues/40476

Indexes would need to change to support the alternative nulls last

create index if not exists priority_idx_%(name)s on %(name)s
    (schedule_at nulls first, (CASE WHEN expected_at IS NULL THEN 1 ELSE 0 END), q_name)
    where dequeued_at is null
          and q_name = '%(name)s';
DETAIL:  source SQL:
create index if not exists priority_idx_queue on queue
    (schedule_at nulls first, (CASE WHEN expected_at IS NULL THEN 1 ELSE 0 END), q_name)
                                                                               ^
HINT:  You have attempted to use a feature that is not yet implemented.
See: https://github.com/cockroachdb/cockroach/issues/9682
retr0h commented 4 years ago

I'll close this ticket as most of the functionality this module relies upon, does not exist yet in cockroach.