Missing Job parameters

[ThMrtns]

Hi,

It seems, that it is currently not possible, to set additional "Job parameters" for the Gluejobs. They are very usefull for example, when additional libraries shall be loaded in the job, but can be used for other things too. It would be great, if it could be configured, e.g. like that:

  Glue:
    jobs:
      - job:
           JobParameters:
              key: value
              key: value

Or is it already possible and I overlooked?

Thanks and best greetings, Th

[toryas]

Hi @ThMrtns ,

You can set additional parameters by DefaulArguments > customArguments example:

Glue:
  ...
  jobs:
    - name: super-glue-job
     ...
      DefaultArguments:
        customArguments: 
          day_partition_key: custom_value
          day_partition_value: other_custom_value

then in your script can acces like this:

import sys
from awsglue.utils import getResolvedOptions

args = getResolvedOptions(sys.argv,
                          ['JOB_NAME',
                           'day_partition_key',
                           'day_partition_value'])
print "The day partition key is: ", args['day_partition_key']
print "and the day partition value is: ", args['day_partition_value']

[ThMrtns]

@toryas Ah, thats awesome! Thank you very much for your reply!