search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.6k stars 3.9k forks source link

(aws-glue): Add scheduler on Glue Jobs #19722

Open emilrowland opened 2 years ago

emilrowland commented 2 years ago

Describe the feature

Add a Glue Trigger Scheduler in the high-level construct for Glue Job.

Link to Glue::Triger CloudFormation: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-glue-trigger.html

Use Case

Would add a simple way to schedule a Glue Job.

Proposed Solution

No response

Other Information

No response

Acknowledgements

CDK version used

2.17.0

Environment details (OS name and version, etc.)

Windows 10

kaizencc commented 2 years ago

I'm labeling this as a p2, which means we don't have this in our immediate roadmap, and it might take time to review a non-trivial PR that introduces this feature. Ideally, we're looking for community support for this (+1s) and can up-level this to a p1 when there's enough support.

wadhah101 commented 2 years ago

I am working on implementing this feature but i am having a hard time on API design for this L2 construct i would appreciate some input

  1. There are 4 trigger types with each one having it's required attribute, for example if the type is SCHEDULED you need the schedule attribute i have the idea of implementing multiple classes for each type with shared abstract class
  2. for the actions attribute , there are two types of actions one that calls glue job and the other calls a crawler , required props differ a lot. Should i implement an abstract class and extend it too ?
  3. Should i focus on a basic L2 construct that does the job and later enhance it with features such as eventbridge intergration , metric , maybe adding helper method sin the glue job L2 construct or do i need a fully featured Construct for a PR ?

References

emilrowland commented 1 year ago

@wadhah101 I believe that a simple L2 construct is enough for now and then it can be enhanced in the future.

daniyar-abdimomunov commented 1 year ago

+1

mijatovicdev commented 1 year ago

+1

michael-stevens commented 1 year ago

+1 this would be useful for us