Support resource detector in auto instrumentation by entry_point or whatever

[wangzlei]

Is your feature request related to a problem? Python sdk supports add resource detector to provider by code change. We want to support custom resource detector in auto-instrumentation without code change..

Describe the solution you'd like load resource detectors from entry_points or by env variable

Describe alternatives you've considered Which alternative solutions or features have you considered? NA

[owais]

Is this for end users or distributors? Could it be solved by implementing a custom configurator that knows which resource detectors to use. opentelemetry-distro implements a configurator here for example: https://github.com/open-telemetry/opentelemetry-python/blob/v1.0.0/opentelemetry-distro/src/opentelemetry/distro/__init__.py#L147

[wangzlei]

Can we follow this pattern https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py#L42 ?

From end users' perspective, auto-instrumentation loads all of resource detectors installed in current environment, unless explicitly excluded by env variable OTEL_PYTHON_DISABLED_RESOURCEDETECTORS

[owais]

Yes, that sounds reasonable if you want this for end users.

[owais]

It is more complicated that that though as the result of resource detectors (resources) must be passed on to the tracer provider constructor. So we'd have to load resource detectors using entry points and pass them to whatever configurator is found. Essentially the configure() method would receive a list of resource detectors found by auto instrumentor.

https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-instrumentation/src/opentelemetry/instrumentation/configurator.py#L48

It'd be up to each configurator to use the detectors when setting up tracer providers. We can update the configurator in opentelemetry-distro to use them.

[ecourreges-orange]

I am not sure if my problem is related, but I can't get any fully auto instrumentation (distro/conf + spans) to work with any other than console_span, i.e. zipkin and jaeger fail Works fine with manual instrumentation, or manual configuration with auto-instrumentation On ubuntu bionic with Python 3.6

~/workspace/auto_instru $ opentelemetry-instrument --trace-exporter zipkin python uninstr.py 
You are using Python 3.6. This version does not support timestamps with nanosecond precision and the OpenTelemetry SDK will use millisecond precision instead. Please refer to PEP 564 for more information. Please upgrade to Python 3.7 or newer to use nanosecond precision.
Configuration of configurator failed
Traceback (most recent call last):
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 99, in _load_configurators
    entry_point.load()().configure()  # type: ignore
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/instrumentation/configurator.py", line 50, in configure
    self._configure(**kwargs)
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/distro/__init__.py", line 153, in _configure
    _initialize_components()
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/distro/__init__.py", line 143, in _initialize_components
    trace_exporters = _import_exporters(exporter_names)
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/distro/__init__.py", line 116, in _import_exporters
    exporter_names, "opentelemetry_exporter"
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/distro/__init__.py", line 97, in _import_tracer_provider_config_components
    selected_component, entry_point_name
RuntimeError: Requested component 'zipkin' not found in entry points for 'opentelemetry_exporter'
Failed to auto initialize opentelemetry
Traceback (most recent call last):
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 110, in initialize
    _load_configurators()
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 103, in _load_configurators
    raise exc
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py", line 99, in _load_configurators
    entry_point.load()().configure()  # type: ignore
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/instrumentation/configurator.py", line 50, in configure
    self._configure(**kwargs)
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/distro/__init__.py", line 153, in _configure
    _initialize_components()
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/distro/__init__.py", line 143, in _initialize_components
    trace_exporters = _import_exporters(exporter_names)
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/distro/__init__.py", line 116, in _import_exporters
    exporter_names, "opentelemetry_exporter"
  File "/home/ecourreges/workspace/auto_instru/lib/python3.6/site-packages/opentelemetry/distro/__init__.py", line 97, in _import_tracer_provider_config_components
    selected_component, entry_point_name
RuntimeError: Requested component 'zipkin' not found in entry points for 'opentelemetry_exporter'
 * Serving Flask app 'uninstr' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:8082/ (Press CTRL+C to quit)

venv properly activated

~/workspace/auto_instru $ which opentelemetry-instrument 
/home/ecourreges/workspace/auto_instru/bin/opentelemetry-instrument

deps installed in venv

~/workspace/auto_instru $ pip list | grep opentel
opentelemetry-api                        1.4.0
opentelemetry-distro                     0.23b0
opentelemetry-exporter-zipkin            1.4.0
opentelemetry-exporter-zipkin-json       1.4.0
opentelemetry-exporter-zipkin-proto-http 1.4.0
opentelemetry-instrumentation            0.23b0
opentelemetry-instrumentation-flask      0.23b0
opentelemetry-instrumentation-requests   0.23b0
opentelemetry-instrumentation-wsgi       0.23b0
opentelemetry-sdk                        1.4.0
opentelemetry-semantic-conventions       0.23b0
opentelemetry-util-http                  0.23b0

Works fine as per the example. If I also install opentelemetry-distro, then I get the span on the console twice (once from code once from auto-distro)

~/workspace/auto_instru $ opentelemetry-instrument --trace-exporter console_span python uninstr.py 
You are using Python 3.6. This version does not support timestamps with nanosecond precision and the OpenTelemetry SDK will use millisecond precision instead. Please refer to PEP 564 for more information. Please upgrade to Python 3.7 or newer to use nanosecond precision.
Overriding of current TracerProvider is not allowed
 * Serving Flask app 'uninstr' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://127.0.0.1:8082/ (Press CTRL+C to quit)
test
127.0.0.1 - - [22/Jul/2021 10:06:26] "GET /server_request?param=test HTTP/1.1" 200 -
{
    "name": "/server_request",
    "context": {
        "trace_id": "0x625cfb2913fd293c335b7c0dc1b1fe1f",
        "span_id": "0x61d2c38df813b52b",
        "trace_state": "[]"
    },
    "kind": "SpanKind.SERVER",
    "parent_id": "0x22eadea02fb81db1",
    "start_time": "2021-07-22T08:06:26.322367Z",
    "end_time": "2021-07-22T08:06:26.335230Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "http.method": "GET",
        "http.server_name": "127.0.0.1",
        "http.scheme": "http",
        "net.host.port": 8082,
        "http.host": "localhost:8082",
        "http.target": "/server_request?param=test",
        "net.peer.ip": "127.0.0.1",
        "http.user_agent": "python-requests/2.26.0",
        "net.peer.port": 56826,
        "http.flavor": "1.1",
        "http.route": "/server_request",
        "http.status_code": 200
    },
    "events": [],
    "links": [],
    "resource": {
        "telemetry.sdk.language": "python",
        "telemetry.sdk.name": "opentelemetry",
        "telemetry.sdk.version": "1.4.0",
        "servce.name": "py-client-gatape-requests",
        "service.name": "py-client-gatape-requests"
    }
}
{
    "name": "/server_request",
    "context": {
        "trace_id": "0x625cfb2913fd293c335b7c0dc1b1fe1f",
        "span_id": "0x61d2c38df813b52b",
        "trace_state": "[]"
    },
    "kind": "SpanKind.SERVER",
    "parent_id": "0x22eadea02fb81db1",
    "start_time": "2021-07-22T08:06:26.322367Z",
    "end_time": "2021-07-22T08:06:26.335230Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "http.method": "GET",
        "http.server_name": "127.0.0.1",
        "http.scheme": "http",
        "net.host.port": 8082,
        "http.host": "localhost:8082",
        "http.target": "/server_request?param=test",
        "net.peer.ip": "127.0.0.1",
        "http.user_agent": "python-requests/2.26.0",
        "net.peer.port": 56826,
        "http.flavor": "1.1",
        "http.route": "/server_request",
        "http.status_code": 200
    },
    "events": [],
    "links": [],
    "resource": {
        "telemetry.sdk.language": "python",
        "telemetry.sdk.name": "opentelemetry",
        "telemetry.sdk.version": "1.4.0",
        "servce.name": "py-client-gatape-requests",
        "service.name": "py-client-gatape-requests"
    }
}

Thanks for any clues.

[ocelotl]

opentelemetry-instrument --trace-exporter zipkin python uninstr.py

Hello

Sorry for the late reply. Please try with opentelemetry-instrument --trace-exporter zipkin_json python uninstr.py instead.

[ecourreges-orange]

opentelemetry-instrument --trace-exporter zipkin python uninstr.py

Hello

Sorry for the late reply. Please try with opentelemetry-instrument --trace-exporter zipkin_json python uninstr.py instead.

Thanks a lot, it works now, so I think I will spend 1/2 day in the next two weeks for a PR of documentation improvement. I now have a functional example with 0 lines of code, a bunch of OTEL_* env vars, and the command opentelemetry-instrument python uninstr.py working as expected.