ThiagoPanini / terraglue

Providing an easy way to deploy a Glue job in any AWS account using Terraform
https://terraglue.readthedocs.io/en/latest/
23 stars 3 forks source link

Not able to call se[BUG] #82

Open mthalla2 opened 3 months ago

mthalla2 commented 3 months ago

:writing_hand: Problem description: ToDo: provide a clear vision of bug reported and point out the impact. Additionaly, insert block codes that help to identify and reproduce the bug.

resource "aws_s3_object" "python_scripts" {

this block is not getting created when using this as source in one of my module like 
module "glue_job" {
  source = "github.com/ThiagoPanini/terraglue"
  glue_job_name = "data_ingestion_glue_test"
  glue_scripts_bucket_name = "psic-etlrepo-uswest2-testing"
  glue_scripts_bucket_prefix = "gluejobs/"
  glue_apply_security_configuration = false
  kms_key_arn = true
  glue_app_dir = "glue"
  subfolders_to_upload = ["python/job_scripts/data_ingestion"]
  glue_main_script_path = "glue/python/job_scripts/data_ingestion/create_catalog_schema_test.py"
  file_extensions_to_upload = [".py"]

....
...
....
}

only glue job is getting created but python files in my directory is not getting pushed to s3

:lady_beetle: Reproducing the problem: To reproduce the bug it's need the take the following steps (example):

  1. [ ] Import module package.module
  2. [ ] Run the function function
  3. [ ] Using paramteres (param_a='a', param_b='b')
  4. [ ] Check the error Exception e

    :gear: Expected behavior: ToDo: describe the feature expected behavior.


    :earth_americas: Additional information: ToDo: if applicable, provide some datails about the environment where the bug showed up. Informations like OS and language version are useful.


    :camera: Screenshots: [OPTIONAL] ToDo: if applicable, provide some screenshots that help to understand the bug.


    :speech_balloon: Possible solutions: [OPTIONAL] Probably the the solution for this bug would be reached through:

    • Trying step A
    • Trying step B
ThiagoPanini commented 3 months ago

Hi @mthalla2! I hope you're doing well.

Thanks for reporting this problem. I will take a look at it as soon as I can. In order to help me debugging this, can you please provide your project/folder structure in which the terraform module is called?