Open fhuthmacher opened 6 months ago
If you rely on manual instrumentation, you should add OTEL Handler to your logging. See example here https://github.com/open-telemetry/opentelemetry-python/blob/main/docs/examples/logs/example.py. If you rely on auto-instrumentation you should set OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED
to true
https://github.com/open-telemetry/opentelemetry-python/blob/da48e0b131ff34ff382b7d1206f71b2e31929cab/opentelemetry-sdk/src/opentelemetry/sdk/environment_variables.py#L587-L598
Thank you, I tried OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED, but that did not change anything.
And when I add the manual instrumentation I get only the below warning in the logs:
"Overriding of current LoggerProvider is not allowed"
so no logs either :/
I couldn't reproduce this. If you can share simple repo with reproduction steps that would be helpful
Hi,
Here is what I did: 1) in my Lambda Dockerfile, I extract the Lambda Layer to /opt
RUN curl $(aws lambda get-layer-version-by-arn --arn arn:aws:lambda:us-east-1:901920570463:layer:aws-otel-python-amd64-ver-1-21-0:1 --query 'Content.Location' --output text) --output layer.zip
RUN unzip layer.zip -d /opt
RUN rm layer.zip
2) Added manual instrumentation to Python code:
import logging
from opentelemetry._logs import get_logger
from opentelemetry._logs import get_logger_provider
from opentelemetry._logs import set_logger_provider
from opentelemetry.exporter.otlp.proto.http._log_exporter import (
OTLPLogExporter,
)
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, SimpleLogRecordProcessor
from opentelemetry.sdk.extension.aws.resource._lambda import (
AwsLambdaResourceDetector,
)
from opentelemetry.sdk.resources import get_aggregated_resources
logger_provider = LoggerProvider(
resource=get_aggregated_resources(
[
AwsLambdaResourceDetector(),
]
),
)
set_logger_provider(logger_provider)
exporter = OTLPLogExporter(endpoint='http://0.0.0.0:4318/v1/logs')
logger_provider.add_log_record_processor(BatchLogRecordProcessor(exporter))
handler = LoggingHandler(level=logging.INFO, logger_provider=logger_provider)
# Attach OTLP handler to root logger
logger = logging.getLogger().addHandler(handler)
# Create different namespaced loggers
loggerAgent = logging.getLogger("financeagent.handler")
loggerAgent.setLevel(os.environ['OTEL_LOG_LEVEL'])
### END
I got the above manual instrumentation to work. But I was not able to get the auto-instrumentation to work. Also how would you set the LogLevel with the auto instrumentation? I wonder if that is what is/was missing?
Here is the repo, which includes the Dockerfile and source code if you want to replicate it (don't deploy the CF template as it deploys a bunch of other things as well).
Also how would you set the LogLevel with the auto instrumentation? I wonder if that is what is/was missing?
It respects the log level of your application settings. It is just another handler registered with global that receives the logs and exports.
Hello,
I have the same problem with logs. The OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
did not help.
I think the logger instrumentation is not present in the lambda distribution https://github.com/open-telemetry/opentelemetry-lambda/blob/main/python/src/otel/otel_sdk/requirements-nodeps.txt
@goyerl
You shouldn't need the logger instrumentation. Logging is part of the OpenTelemetry SDK.
@fhuthmacher
Is this still an issue?
I am trying to get to work traces, metrics and logs via AWS Distro for OpenTelemetry Collector (ADOT Collector), using the following Lambda Layer: arn:aws:lambda:us-east-1:901920570463:layer:aws-otel-python-amd64-ver-1-20-0:3 and Python 3.8.
My understanding is that this version includes OpenTelemetry Python v1.20.0 and ADOT Collector v0.35.0 and should support, trace, metrics and logs as experimental. The issue seems to be that no matter how I configure otlphttp, the collector does not seem to export logs at all.
Steps to reproduce 1) create Python 3.8 Lambda 2) Add Layer arn:aws:lambda:us-east-1:901920570463:layer:aws-otel-python-amd64-ver-1-20-0:3 3) create collector.yml file with below content in root dir:
4) Set two environment variables: AWS_LAMBDA_EXEC_WRAPPER = /opt/otel-instrument OPENTELEMETRY_COLLECTOR_CONFIG_FILE = /var/task/collector.yml
5) Update lambda_function.py with below code:
What is the expected behavior? I would expect to see a similar output in logging for "logs" to what I get for traces and metrics.
What is the actual behavior? I don't see any log output, only traces and metrics, see below example.