aws-samples / aws-glue-samples

AWS Glue code samples
MIT No Attribution
1.44k stars 820 forks source link

Adding ability to (de)serialize visual DAG and table information v2 #163

Closed andreivmaksimov closed 1 month ago

andreivmaksimov commented 1 month ago

Issue #, if available: NA

Description of changes: Adding ability to serialize and deserialize jobs information

Usage example:

  1. Serialize jobs: python sync.py --src-role-arn SRC_ACCOUNT_IAM_ROLE_ARN --src-region us-east-1 --serialize-to-file resources.json --targets job,catalog --skip-prompt
  2. Deploy Glue jobs configuration from file: python sync.py --dst-role-arn TARGET_ACCOUNT_IAM_ROLE_ARN --dst-region us-east-1 --deserialize-from-file resources.json --config-path mapping.json --targets job,catalog --skip-prompt

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Testing:

All tests are executed from the Cloud Shell in the testing AWS account.

moomindani commented 1 month ago

Thank you for your contribution!