Opencensus config in uWSGI based container?

[dabedin]

Versions:

• python: 3.7.3
• Flask: 1.0.2
• OpenCensus modules:
• opencensus 0.5.0
• opencensus-context 0.1.0
• opencensus-ext-azure 0.1.0
• opencensus-ext-flask 0.3.0
• opencensus-ext-requests 0.1.2
• opencensus-ext-threading 0.1.2

My app.py OpenCensus config:

exporter=AzureExporter()
sampler=ProbabilitySampler(1.0)
middleware = FlaskMiddleware(app, 
    exporter=exporter,
    sampler=sampler,
    blacklist_paths=['healthz'])

API requests received by my Flask sample flows via OpenCensus to the exporter (AzureExporter) without any issue, while debugging/launching the .py directly. The Flask API is meant to be based on tiangolo/uwsgi-nginx-flask:python3.7 image: once I test it in Docker no API requests seems to flow via OpenCensus. The opencensus-ext-azure specific env variables gets passed to the container. Not understanding the root cause, I tested the same OpenCensus modules based on python:3.7 image: all requests flows to my exporter. Is there any uWSGI configuration that might interact with OpenCensus and the Flask extension? My uwsgi.ini:

[uwsgi]
module = app
callable = app
uid = 1000
master = true
enable-threads = true

[blairg23]

You may need to handle the pre-fork nature of uWSGI and force post-fork event sending using uwsgidecorators like postfork. We had to do something similar when enabling tracing for Honeycomb.io. In fact, we had to do it with Celery as well (and gunicorn would have same problem as uWSGI.

[iMichka]

This has bitten me recenlty. Opencensus was working locally when running with Flask only. As soon as I ran Flask + uwsgi inside a docker container, the logs were not sent to azure anymore.

Here is an explanation: https://engineering.ticketea.com/uwsgi-preforking-lazy-apps/

In short (and if I understood correctly), opencensus sets up an extra thread to send the logs/metrics. Once uwsgi forks the processes, that thread is not copied to the new processes, which breaks the sending of the logs/metrics.

Using lazy-apps = true in my uwsgi.ini file fixed it. I'll have a look at the postfork solution too later.

This definitvely needs some more documentation, and a big red warning in the documentation.

[llaagg]

And it's in now in ms docs ms docs app insights

[symphoton]

A great explanation of the relevant uwsgi options (lazy-apps, enable-threads, master, single-interpreter) can be found here: https://docs.newrelic.com/docs/apm/agents/python-agent/web-frameworks-servers/python-agent-uwsgi-web-server/ @llaagg Maybe it's worth to mention the other options in ms docs as well?