Opentelemetry stops sending traces with 9.0.0

[moserke]

Summary

When going from 8.6.2 to 9.0.0 the opentelemetry callback stops sending traces to the endpoint. Same exact configuration and traces get forwarded in 8.6.2 but go nowhere in 9.0.0. I suspect it's due to how the exporter is getting picked but can't seem to figure out how to make it work.

otel_exporter = None
        if store_spans_in_file:
            otel_exporter = InMemorySpanExporter()
            processor = SimpleSpanProcessor(otel_exporter)
        else:
            if otel_exporter_otlp_traces_protocol == 'grpc':
                otel_exporter = GRPCOTLPSpanExporter()
            else:
                otel_exporter = HTTPOTLPSpanExporter()
            processor = BatchSpanProcessor(otel_exporter)

Issue Type

Bug Report

Component Name

opentelemetry callback

Ansible Version

$ ansible --version
ansible [core 2.17.1]
  config file = /ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.12/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.12.3 (main, Apr 17 2024, 00:00:00) [GCC 14.0.1 20240411 (Red Hat 14.0.1-0)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True

Community.general Version

$ ansible-galaxy collection list community.general
Collection        Version
----------------- -------
community.general 9.1.0  

Configuration

$ ansible-config dump --only-changed

OS / Environment

No response

Steps to Reproduce

ansibile config: [defaults] callbacks_enabled = community.general.opentelemetry

Run playbook OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 ansible-playbook playbook.yml

Expected Results

Expect traces to be sent to endpoint

Actual Results

Traces are never forwarded

Code of Conduct

• [X] I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:

• plugins/callback/opentelemetry.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibullbot]

cc @v1v click here for bot help

[v1v]

https://github.com/ansible-collections/community.general/pull/8321 is the PR that introduced the support for the http exporter.

As far as I see, the change uses the same exporter by default.

Can you try to run the plugin with the explicit configuration entries?

ansible.cfg:

    [defaults]
    callbacks_enabled = community.general.opentelemetry
    [callback_opentelemetry]
    otel_exporter_otlp_traces_protocol = grpc
    store_spans_in_file = None

IIUC, you tried locally running OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317/ ansible-playbook playbook.yml against your OTEL collector, right?

[moserke]

Tried setting all of the possible config options to their defaults in the ansible.cfg and still the same issue, it just simply isn't trying to send the traces. If I do a store_spans_in_file=/dev/stdout instead just to see, it prints them to the screen, so I know it's tracing, it's just for some reason not sending to the otlp endpoint...