search

seung-lab / python-task-queue

Asynchronous serverless task queue with timed leasing of tasks. Threaded implementations for SQS and local filesystem.
BSD 3-Clause "New" or "Revised" License
36 stars 9 forks source link

feat(api): GCP Pub/Sub #22

Open nkemnitz opened 4 years ago

nkemnitz commented 4 years ago

PR adds support for Google's Pub/Sub queue

TODO: Add tests and maybe look into the missing features

william-silversmith commented 4 years ago

Ah, I seem to have missed this. Is this Google's primary task queue? Seems like they also have https://cloud.google.com/tasks/

william-silversmith commented 4 years ago

Seems like pubsub is more like what we want. https://cloud.google.com/tasks/docs/comp-pub-sub

nkemnitz commented 4 years ago

Hmm, I think Pub/Sub is more similar to SQS than Cloud Tasks, but there is no 100% perfect match between Pub/Sub, Cloud Tasks, SQS and SNS...

Notable differences between Pub/Sub and SQS I remember I ran into: Pub/Sub allows huge payloads (10 MB), but the maximum lease time limit is much shorter (10 minutes, iirc).

william-silversmith commented 4 years ago

I'm going to try to integrate this, but I'll probably have to do some radical surgery to this PR or open a new one due to the major refactoring that just happened.

william-silversmith commented 3 years ago

Would you still be interested in integrating this Nico? Happy to do it, just want to make sure there's interest.