search

open-telemetry / opentelemetry-cpp

The OpenTelemetry C++ Client
https://opentelemetry.io/
Apache License 2.0
810 stars 391 forks source link

[EXPORTER] OTLP HTTP session canceled after a flush, with async export #2715

Open TheodoreG opened 6 days ago

TheodoreG commented 6 days ago

Describe your environment Hey guys, I ran into the same problem wit https://github.com/open-telemetry/opentelemetry-cpp/issues/1955.In my side, some error ocassionally occured, and it may be due to http response latency described in https://github.com/open-telemetry/opentelemetry-cpp/issues/1955. Can someone helps, thanks.

version v1.15.0 cmake command: CC=gcc-11 CXX=g++11 cmake .. -DBUILD_TESING=ON -DWITH_OTLP_HTTP=ON -DOTELCPP_PROTO_PATH=XXXX/opentelemetry-proto-1.2.0 -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Release -DOPENTELEMETRY_INSTALL=ON -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DWITH_ASYNC_EXPORT_PREVIEW=ON

Steps to reproduce ./examples/otlp/example_otlp_http 0.0.0.0:4318/v1/traces

otel collector config 92896800-9e79-48cd-b0f5-8610f6d9949d

What is the expected behavior? Should be no error.

What is the actual behavior? Print error: 67894df3-96d3-47b4-96ec-f68e7f2a7f4d

Additional context it seems I have the same problem with https://github.com/open-telemetry/opentelemetry-cpp/issues/1955. I find the issue is closed while the located code has no change.

marcalff commented 6 days ago

Thanks for the report.

Using opentelemetry-1.16.0 with WITH_ASYNC_EXPORT_PREVIEW=OFF: can not reproduce

Using opentelemetry-1.16.0 with WITH_ASYNC_EXPORT_PREVIEW=ON: reproduced

The example code example_otlp_http invokes ForceFlush(), which is supposed to wait for spans to be exported.

It seems ForceFlush() is not working for async export, to investigate.

Note that related issue #1955 was fixed by #2000, saying it was closed without being fixed is incorrect.

owent commented 5 days ago

Thanks for the report.

Using opentelemetry-1.16.0 with WITH_ASYNC_EXPORT_PREVIEW=OFF: can not reproduce

Using opentelemetry-1.16.0 with WITH_ASYNC_EXPORT_PREVIEW=ON: reproduced

The example code example_otlp_http invokes ForceFlush(), which is supposed to wait for spans to be exported.

It seems ForceFlush() is not working for async export, to investigate.

Note that related issue #1955 was fixed by #2000, saying it was closed without being fixed is incorrect.

The problem is providers will forceflush and then shutdown when destroying before, and them only will shutdown now. I will change the behavious of http client to call forceflush first to solve this problem.