FileExistsError: [Errno 17] File exists: '/root/.opencensus/.azure/gunicorn' when I try to run Docker

[renan-alonkin]

Hello there,

I'm currently having the following problem: I have an API, and I want to use Opencensus to export my exceptions to Azure's Application Insights, but, when I try to run it inside of a Docker Container this problem appears:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
...

File "/app/opensensus_mid.py", line 39, in <module>
    connection_string=connection_string))
  File "/usr/local/lib/python3.7/site-packages/opencensus/ext/azure/log_exporter/__init__.py", line 127, in __init__
    retention_period=self.options.storage_retention_period,
  File "/usr/local/lib/python3.7/site-packages/opencensus/ext/azure/common/storage.py", line 90, in __init__
    self._maintenance_routine(silent=False)
  File "/usr/local/lib/python3.7/site-packages/opencensus/ext/azure/common/storage.py", line 112, in _maintenance_routine
    os.makedirs(self.path)
  File "/usr/local/lib/python3.7/os.py", line 221, in makedirs
    mkdir(name, mode)
FileExistsError: [Errno 17] File exists: '/root/.opencensus/.azure/gunicorn'

Describe your environment.

• Python 3.8.0 (default, Nov 6 2019, 16:00:02)
• Docker:

  
  Client:
  Version:      17.03.1-ce-rc1
  API version:  1.27
  Go version:   go1.7.5
  Git commit:   3476dbf
  Built:        Wed Mar 15 20:33:22 2017
  OS/Arch:      windows/amd64

Server: Version: 17.03.1-ce-rc1 API version: 1.27 (minimum version 1.12) Go version: go1.7.5 Git commit: 3476dbf Built: Wed Mar 15 20:28:18 2017 OS/Arch: linux/amd64

- Opencensus `opencensus==0.7.5
opencensus-ext-azure==1.0.0`

- FastAPI: `fastapi==0.42.0
 uvicorn==0.10.8
gunicorn==20.0.0`

**Steps to reproduce.**
I've created an "Opencensus middleware" for FastAPI, the code can be found at [this link](https://pastebin.com/MyakznYk). All I do is call it inside FastAPI middleware. 

@app.middleware("http") async def opencensus_tracking(request: Request, call_next):

Creates Tracer and Span (Request)

opencensus_instance._before_request(request=request)

# Execute the calling method and then get the result
response = await call_next(request)

# Send the status of `responce`
opencensus_instance._after_request(response=response)

# Closes the Span and finish the trace
opencensus_instance.tracer_instance.end_span()
opencensus_instance.tracer_instance.finish()
return response

---------------------------
# UPDATE: 

Okay, I've found the reason that this error is occuring, but I don't know how to solve it (completely). 

This problem occurs because I'm trying to run opencensus in a multi-worker env. To solve it, I changed `os.makedirs(self.path)` to `os.makedirs(self.path, exist_ok=True)`, but `exist_ok=True` is not compatible with Python 2. Can anybody help me to solve it? I would like to make a PR to this solution. 

[jeremy010203]

Hi @RenanAlonkin , Got the same issue on my docker webapp on Azure.

I have found a workaround, you can specify a storage_path when creating the AzureExporter object:

AzureExporter(
    connection_string=f'InstrumentationKey={telemetry_key}',
    storage_path=os.path.join(
        os.path.expanduser('~'),
        '.opencensus',
        '.azure',
        str(uuid.uuid4()),
        os.path.basename(sys.argv[0]) or '.console',
    )
)

It creates a folder with unique id per thread in the .azure folder (inspired from the default value of storage_path). I am not sure that uuid4 is a good solution but it seems to work just fine.

[lzchen]

@RenanAlonkin Are you still wanting [#817] to be merged?

[renan-alonkin]

To be honest, no, I've created a workaround for my project. It does not look good, but it works. What jeremy posted looks like it will work, when I have time I will try to use it.

[jeremy010203]

Using the azure ext for multiple threads/workers applications is a common pattern for web applications I think. I had no problem with the appinsights python sdk but it is deprecated now so I had to switch my app to opencensus for logging.

It would be nice to have a fix as more and more people are going to migrate to this library.

PS: if two azure exporters can live together on the same .azure folder maybe the solution of @RenanAlonkin is better but I did not spend enough time on the code to say

[lzchen]

Specifying the storage path is the recommended approach for this issue. Closing.

[lzchen]

Re-opening to implement a more permanent fix. See [#903].

[jeremy010203]

Hi @lzchen , Is the storage path thread safe? Because as I understand it tmpdir+opencensus+ikey is constant (Maybe we will same the same issue)

[lzchen]

@jeremy010203 Good question. Changing the directory was not for fixing this issue. The SDK simply does not throw an error now if the folder already exists.