Duplicate log record on setting basic config

[king-11]

• Package Name: azure-monitor-opentelemetry
• Package Version: ^1.6.0
• Operating System: Mariner
• Python Version: 3.9.7

Describe the bug I am seeing duplicate logs in AKS for the python application after enabling formatting on the logger that I create for each module. There are two logs for each logging call in the AKS container logs one with default format and one with customized format that I defined.

To Reproduce Create logger using below code:

configure_azure_monitor(
      enable_live_metrics = True,
      disable_offline_storage = True
  )

def get_logger(name: str, level = logging.INFO):
    logger = logging.getLogger(name)
    logger.setLevel(level)
    logging.basicConfig(
        format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
    )
    return logger

logger = get_logger(__name__)

logger.info("hello suspicious duplicate")

The other approach I tried was modifying get_logger as below:

console_handler = logging.StreamHandler(sys.stdout)
console_formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s : %(message)s")
console_handler.setFormatter(console_formatter)
logger.addHandler(console_handler)

This also results in duplicate logs

Expected behavior Only my custom log should get printed

[github-actions[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @gracewilcox @gulopesd @Haiying-MSFT @jairmyree @joshfree @KarishmaGhiya @KevinBlasko @kurtzeborn @nisha-bhatia @pvaneck @sarangan12 @scottaddie @srnagar @ToddKingMSFT.

[lzchen]

@king-11

Please pass in the name of the Python logger you would like to track telemetry for like in this example. Keep in mind that every logger that is in the same namespace as the name passed in will also be tracked.

i.e., if I use logger_name="azure.app", the loggers with example names azure.app.test, azure.app.webapp and azure.app.function will be tracked but azure.testing will not.

[king-11]

@lzchen passing in the logger_name="anomaly_detection" which is the parent package name also results in duplicate logging. I was just wondering if there is any way to format the log using opentelemetry distro as I think two stream handler are being created here on using the format options I have tried.

[king-11]

adding more details which I recently found the two log records being generated differ in details apart from the message

sdkVersion	customDimensions	operation_Id	client_Browser	cloud_RoleName
ulm_py3.9.19:otel1.25.0:ext1.0.0b27	{"code.filepath":"/usr/lib/python3.9/site-packages/anomaly_detection/handlers/inference_handler.py","code.function":"execute_inference","code.lineno":"105"}	ecd6e2a7c786da27c0e3097b37d8c6df	Other	impact-service
py3.9.19:oc0.11.4:ext1.1.13	{"process":"MainProcess","module":"inference_handler","fileName":"/usr/lib/python3.9/site-packages/anomaly_detection/handlers/inference_handler.py","lineNumber":"105","level":"INFO"}	00000000000000000000000000000000	Python Requests 2.32	main.py

cloud role name is incorrect for the later sdk which is the duplicate record being generated

[lzchen]

@king-11

The second sdk is for the opencensus python sdk which will be deprecated in September.

[king-11]

@lzchen so it comes along with azure monitor distro is it? Because I remember removing it when migrating to open telemetry.

[lzchen]

@king-11

It does not come with the distro. Opencensus should be completely separate from Open Telemetry. Please make sure you are not using the old opencensus sdk and that you are not querying old data from before your testing. Also please try out your app outside of an AKS environment to see if the issue persists.

[github-actions[bot]]

Hi @king-11. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[github-actions[bot]]

Hi @king-11, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

[king-11]

@lzchen so I confirmed that opencensus dependency was still present in our setup.py but it isn't being imported or used anywhere. So is it possible that just the presence of this particular package in system environment causes dual export of logs?

[lzchen]

@king-11

opencensus packing existing but not being used in your code should not cause export of logs from the opencensus azure monitor sdk. Please make sure that you are not using it anywhere in your code. I would also uninstall the package as it is deprecated.

[king-11]

Closing this issue thanks for help @Izchen one of our dependencies was pulling in opencensus and using it for creating an exporter.