Importing `opentelemetry.instrumentation.aws_lambda` module is slow because of `pkg_resources`

[serkan-ozal]

Describe your environment

Environment: AWS Lambda Python version: Python 3.9 Package version: 0.47b0

What happened?

Initialization of the OTEL Python Lambda layer takes significant amount of time in the AWS Lambda environment because of module imports. When I have analyzed import timings, I have noticed that importing opentelemetry.instrumentation.aws_lambda is slow, because of opentelemetry.instrumentation.instrumentor > opentelemetry.instrumentation.dependencies > pkg_resources imports as shown below.

Can we make importing opentelemetry.instrumentation.dependencies in the instrumentor.py lazy and optional (ignore when skip_dep_check arg is True) here: https://github.com/open-telemetry/opentelemetry-python-contrib/blob/98958b6f1ef09d332e13ea1d7bc308f8e9edd351/opentelemetry-instrumentation/src/opentelemetry/instrumentation/instrumentor.py#L84 So we can configure/optimize it in the AWS Lambda Python layer side.

Portion of the module import timings related to opentelemetry.instrumentation.aws_lambda module import in a Lambda function with 512 MB memory.

import time:      3143 |       3143 |         pkgutil
import time:       437 |        437 |               xml
import time:       356 |        793 |             xml.parsers
import time:       700 |        700 |             pyexpat
import time:       326 |       1817 |           xml.parsers.expat
import time:      1521 |       3338 |         plistlib
import time:      1121 |       1121 |           packaging
import time:       677 |        677 |                     sysconfig
import time:      1395 |       1395 |                       packaging._elffile
import time:      2261 |       3655 |                     packaging._manylinux
import time:      1225 |       1225 |                     packaging._musllinux
import time:      3802 |       9357 |                   packaging.tags
import time:       691 |        691 |                     packaging._structures
import time:      5171 |       5861 |                   packaging.version
import time:      2460 |      17677 |                 packaging.utils
import time:     11271 |      28948 |               packaging.specifiers
import time:      7844 |      36791 |             packaging._tokenizer
import time:      2756 |      39546 |           packaging._parser
import time:      3292 |      43958 |         packaging.markers
import time:       974 |        974 |         packaging.requirements
import time:       946 |        946 |           jaraco
import time:      4772 |       4772 |                 more_itertools.recipes
import time:     22981 |      27752 |               more_itertools.more
import time:       472 |      28223 |             more_itertools
import time:      2285 |      30508 |           jaraco.functools
import time:      1831 |       1831 |             backports
import time:       312 |        312 |                 backports.tarfile.compat
import time:      1558 |       1870 |               backports.tarfile.compat.py38
import time:     21746 |      23615 |             backports.tarfile
import time:      2868 |      28313 |           jaraco.context
import time:      4887 |      64653 |         jaraco.text
import time:      4140 |       4140 |           platformdirs.api
import time:       357 |        357 |           platformdirs.version
import time:      2635 |       2635 |             configparser
import time:      1841 |       4476 |           platformdirs.unix
import time:      4259 |      13230 |         platformdirs
import time:     66019 |     195313 |       pkg_resources
import time:      2064 |     197376 |     opentelemetry.instrumentation.dependencies
import time:      1511 |     248706 |   opentelemetry.instrumentation.instrumentor
import time:      2923 |     252130 | opentelemetry.instrumentation.aws_lambda

Steps to Reproduce

• Set PYTHONPROFILEIMPORTTIME env var to true in the otel-instrument bash script
• Import opentelemetry.instrumentation.aws_lambda module

Expected Result

-

Actual Result

-

Additional context

No response

Would you like to implement a fix?

None

[xrmx]

@serkan-ozal What do you think of reconsider things after https://github.com/open-telemetry/opentelemetry-python-contrib/pull/2181 ?

[serkan-ozal]

Hi @xrmx, Yes, that PR makes sense. Of course it is better if we could switch from pkg_resources. I have verified that importing importlib.metadata has negligible effect on Lambda coldstart.

Looking forward to get it approved, merged and released.