Open jankatins opened 2 years ago
This seems to be modeled closely after the CloudFormation API which seems to me only supports this option in defaultArguments
We could potentially expose this in a more convenient way through our API doing something similar to your suggestion. However I'm not certain that's the best thing for us to do, personally I'm not super familiar with Glue as a service or how people typically define a Job in CloudFormation
The team won't be able to get to this, but we do accept contributions! Check out our contributing guide if you're interested - we'd be happy to review a PR
Just to clarify: I meant the the aws_glue_alpha package with the L2 constructs.
There is already this:
executable=glue.JobExecutable.python_etl(
...
extra_python_files=[glue_additional_python_files],
),
results in these two default args:
"DefaultArguments": {
"--job-language": "python",
"--extra-py-files": extra_py_files,
I've no experience with CF, but the current way results in an empty field in the glue UI, which seems wrong, given that the default for a new glue job created in the UI is a "Enable" and there is no way to actually clear that field.
Yes I'm aware you meant our alpha package 🙂
I mention CloudFormation because if our only option for specifying this is within CloudFormation's DefaultArguments
then that will impact our design and implementation of this potential feature in the L2
I find the mismatch between "add it as a default argument" (which is also used for passing in arguments to the job script itself) and "it has an extra drop down in the UI instead of an simple input field, like the arguments for your job" very strange and I'm actually looking forward to have a proper API for all these interesting switches :-)
Following. Would be nice to have this
Describe the feature
Enabling or disabling job bookmarks is an important part of a glue job. Unfortunately, it seems this is not yet supported by the
glue.Job
in a nice way, only viadefault_arguments
.Use Case
It's a big feature of glue jobs, that you can resume from where you left off.
Proposed Solution
Add a new flag somewhere which takes an enum with the required three values and add the appropriate flag into the default arguments.
Other Information
No response
Acknowledgements
CDK version used
2.40.0 (build 56ba2ab
Environment details (OS name and version, etc.)
mac