search

Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
6.38k stars 4.26k forks source link

ImportError: cannot import name '_set_http_host' from 'opentelemetry.instrumentation._semconv' #2099

Open cforce opened 1 month ago

cforce commented 1 month ago

Since i switched on app insight opentelemetry dependency incompatibilities fights back. I still don't have an explanation as version opentelemetry-instrumentation==0.47b0 used in requirements.txt definitely has a method "set_http_host"

ImportError: cannot import name '_set_http_host' from 'opentelemetry.instrumentation._semconv' (/agents/python/opentelemetry/instrumentation/_semconv.py)

https://github.com/Azure-Samples/azure-search-openai-demo/blob/e34edd561b575565e686aed7a0eb125ce8215a6d/app/backend/requirements.txt#L230

https://github.com/open-telemetry/opentelemetry-python-contrib/blob/07c3324a3bcf80ee716f90b2456f59af08b685fa/opentelemetry-instrumentation/src/opentelemetry/instrumentation/_semconv.py#L329

The import is used at app.py:35

"from opentelemetry.instrumentation.httpx import ( HTTPXClientInstrumentor, )

What makes me curios is

ImportError: cannot import name '_set_http_host' from 'opentelemetry.instrumentation._semconv' (/agents/python/opentelemetry/instrumentation/_semconv.py)

It looks like it tries to load the module froma by Azure at runtime injected pypath which is set by an Agent. So this agnet my not use at all the proper version opentelemetry-instrumentation==0.47b0 .

2024-10-30T22:49:46.200141546Z Traceback (most recent call last):
2024-10-30T22:49:46.200149647Z   File "/opt/python/3.11.8/lib/python3.11/site-packages/gunicorn/arbiter.py", line 609, in spawn_worker
2024-10-30T22:49:46.200154747Z     worker.init_process()
2024-10-30T22:49:46.200158747Z   File "/tmp/8dcf930cbcef97b/antenv/lib/python3.11/site-packages/uvicorn/workers.py", line 75, in init_process
2024-10-30T22:49:46.200164847Z     super().init_process()
2024-10-30T22:49:46.200168747Z   File "/opt/python/3.11.8/lib/python3.11/site-packages/gunicorn/workers/base.py", line 134, in init_process
2024-10-30T22:49:46.200172747Z     self.load_wsgi()
2024-10-30T22:49:46.200176547Z   File "/opt/python/3.11.8/lib/python3.11/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
2024-10-30T22:49:46.200180947Z     self.wsgi = self.app.wsgi()
2024-10-30T22:49:46.200184947Z                 ^^^^^^^^^^^^^^^
2024-10-30T22:49:46.200189047Z   File "/opt/python/3.11.8/lib/python3.11/site-packages/gunicorn/app/base.py", line 67, in wsgi
2024-10-30T22:49:46.200193347Z     self.callable = self.load()
2024-10-30T22:49:46.200197347Z                     ^^^^^^^^^^^
2024-10-30T22:49:46.200201047Z   File "/opt/python/3.11.8/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
2024-10-30T22:49:46.200205547Z     return self.load_wsgiapp()
2024-10-30T22:49:46.200210447Z            ^^^^^^^^^^^^^^^^^^^
2024-10-30T22:49:46.200215147Z   File "/opt/python/3.11.8/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
2024-10-30T22:49:46.200219647Z     return util.import_app(self.app_uri)
2024-10-30T22:49:46.200224247Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-30T22:49:46.200228347Z   File "/opt/python/3.11.8/lib/python3.11/site-packages/gunicorn/util.py", line 371, in import_app
2024-10-30T22:49:46.200232947Z     mod = importlib.import_module(module)
2024-10-30T22:49:46.200237547Z           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-30T22:49:46.200242347Z   File "/opt/python/3.11.8/lib/python3.11/importlib/__init__.py", line 126, in import_module
2024-10-30T22:49:46.200247147Z     return _bootstrap._gcd_import(name[level:], package, level)
2024-10-30T22:49:46.200251448Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-30T22:49:46.200255648Z   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
2024-10-30T22:49:46.200260048Z   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
2024-10-30T22:49:46.200264048Z   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
2024-10-30T22:49:46.200279448Z   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
2024-10-30T22:49:46.200284248Z   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
2024-10-30T22:49:46.200288748Z   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
2024-10-30T22:49:46.200294848Z   File "/tmp/8dcf930cbcef97b/main.py", line 3, in <module>
2024-10-30T22:49:46.200299848Z     from app import create_app
2024-10-30T22:49:46.200304048Z   File "/tmp/8dcf930cbcef97b/app.py", line 35, in <module>
2024-10-30T22:49:46.200308248Z     from opentelemetry.instrumentation.httpx import (
2024-10-30T22:49:46.200312148Z   File "/tmp/8dcf930cbcef97b/antenv/lib/python3.11/site-packages/opentelemetry/instrumentation/httpx/__init__.py", line 199, in <module>
2024-10-30T22:49:46.200316748Z     from opentelemetry.instrumentation._semconv import (
2024-10-30T22:49:46.200321148Z ImportError: cannot import name '_set_http_host' from 'opentelemetry.instrumentation._semconv' (/agents/python/opentelemetry/instrumentation/_semconv.py)
cforce commented 1 month ago

I disabled AppInsighs and restarted the server and now it works again.

pamelafox commented 4 weeks ago

I saw that as well, and I've reported it to the App Service team. That's why I have not yet merged https://github.com/Azure-Samples/azure-search-openai-demo/pull/2083 This is only with the optional variable, right? Not with the default deployment?

cforce commented 4 weeks ago

I have manually activated in Portal, which is the same result as if I would use.enabled in bicep I assume

pamelafox commented 4 weeks ago

Okay, thanks for clarifying, I've reached out again to the App Service team.

jeremydvoss commented 3 weeks ago

Yes, disabling autoinstrumentation in the portal is the correct response. It is causing a dependency conflict with the Manual Instrumentation built into this app.

This AI demo includes "azure-monitor-opentelemetry==1.6.1" and enables "Manual instrumentation" with this distro here so long as you set the APPLICATIONINSIGHTS_CONNECTION_STRING. You should only use Manual Instrumentation or Autoinstrumentation, but not both. See this note in our documentation: Only use autoinstrumentation on App Service if you aren't using manual instrumentation of OpenTelemetry in your code...

Since this test app already uses manual instrumentation, I recommend that anyone who has this issue disable autoinstrumentation. You can do this in the "Application Insights" blade of App Service (or, if you enabled this feature in an ARM template, make sure ApplicationInsightsAgent_EXTENSION_VERSION is set to "disabled").