Error with 3.7.0: Unable to import module 'lambda_function': cannot import name '_rand' from 'ddtrace.internal'

[chrisforrette]

Describe the bug

Hey there, so I recently attempted an upgrade of this module to the latest version (3.7.0) from 3.1.1 and the forwarder Lambda started failing and spitting out these logs:

[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': cannot import name '_rand' from 'ddtrace.internal' (/var/task/ddtrace/internal/__init__.py) 

I was previously pinned to version 3.1.1 of this module and 3.40.0 of the Datadog forwarder and tried to update to 3.7.0 and 3.44.0, respectively. Though if I pin to 3.7.0 and forwarder version 3.40.0, I see this in the plan and it still breaks:

  ~ resource "aws_lambda_function" "this" {
      ~ architectures                  = [
          + "x86_64",
        ]
        id                             = "preprod-datadog-logs-forwarder"
      ~ last_modified                  = "2022-03-23T17:50:35.000+0000" -> (known after apply)
      ~ runtime                        = "python3.7" -> "python3.9"
        tags                           = {
            "DD_FORWARDER_VERSION" = "3.40.0"
            "Env"                  = "preprod"
        }
        # (20 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

I'm guessing this has to do with the architecture and Python runtime updates introduced in v3.5.0, and I found your GitHub issue with the exact same log over here: https://github.com/DataDog/datadog-lambda-python/issues/88 ...but I'm not clear on how to remediate and I'd love any insight or suggestions you might have as it seems you've dug into it pretty deep.

To Reproduce This is my old configuration:

module "datadog_log_forwarder" {
  source = "clowdhaus/datadog-forwarders/aws"
  version = "3.1.1"

  dd_api_key            = ""
  dd_api_key_secret_arn = var.api_key_secret_arn
  kms_alias             = var.kms_key_alias

  log_forwarder_version = "3.40.0"
  log_forwarder_name    = "${var.environment_name}-datadog-logs-forwarder"
  log_forwarder_environment_variables = {
    DD_TAGS = "env:${local.dd_env}"
  }
  bucket_name             = "my-bucket-name"
  create_vpc_fl_forwarder = false
}

And this is the new configuration that broke, though the log_forwarder_version can be set to 3.40.0 and the error still occurs:

module "datadog_log_forwarder" {
  source = "clowdhaus/datadog-forwarders/aws"
  version = "3.7.0"

  dd_api_key            = ""
  dd_api_key_secret_arn = var.api_key_secret_arn
  kms_alias             = var.kms_key_alias

  log_forwarder_version = "3.44.0"
  log_forwarder_name    = "${var.environment_name}-datadog-logs-forwarder"
  log_forwarder_environment_variables = {
    DD_TAGS = "env:${local.dd_env}"
  }
  bucket_name             = "my-bucket-name"
  create_vpc_fl_forwarder = false
}

Expected behavior

The forwarder Lambda function continues to run without error

Thanks!

[bryantbiggs]

Closed with #17

[chrisforrette]

@bryantbiggs @Vrtak-CZ Thank you!

[jhrcek]

Hello, I just got the same error while using 4.1.0 version of this module. I can still see references to python3.9 here (which I assume is what caused my deployment to use python 3.9) https://github.com/terraform-aws-modules/terraform-aws-datadog-forwarders/blob/c7e6d1995b0aabd98557932c7735d08c285777ed/variables.tf#L208 . Should these occurrences be changed to 3.7 too?

[Vrtak-CZ]

@jhrcek thats probably because we are using only log_forwarder submodule. So I updated config only for log_forwarders not for the whole module. Datadog says the forwarders should run with Python 3.7 so the default for all forwarders should be 3.7.

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.