mooyoul
commented
7 years ago checked feature availabilities:
Lambda + SQS + SNS + CloudWatch Logs + CloudWatch Events (schedule)
- [x] Delayed jobs
- This is natively supported feature in SQS service. We will use delayed messsage to handle delayed jobs.
- [x] Distribution of parallel work load
- Lambda + SQS might be utilize maximum distribution of workload
- [x] Job event and progress pubsub
- Job event / progress pubsub can be implemented via SNS
- IMO it should be implemented via additional plugin because SNS requests are not free even cheap its cost.
- [x] Job TTL
- IMO Job TTL can be handled via Lambda function timeout.
- [x] Optional retries with backoff
- This is natively supported feature with async invocation of lambda function if DLQ (Dead letter queue) or SNS was specified on target lambda function.
- [ ] Graceful workers shutdown
- I don't have any idea yet.
- [ ] Full-text search capabilities
- I don't have any idea yet.
- [x] RESTful JSON API
- Queue status
- Job management (create, update, delete, modify)
- API Gateway + Lambda?
- [x] Rich integrated UI
- yeah this could be implemented with REST API
- [x] Infinite scrolling
- easy.
- [ ] UI progress indication
- I don't have any idea yet.
- [x] Job specific logging
- IMO Job specific logging exactly same as AWS CloudWatch Logs.
- [ ] Powered by Redis
- I think that Jobless would be nice if it provides cheap aws cost as possible.
- We only need storage that can handle job metadata with atomic operation.
- SimpleDB seems deprecated, DynamoDB is hard to estimate unit cost, Elasticache is not cheap.
- Use S3 as storage?
Below features are came from Kue. since this project was highly inspired by Kue.