elastic / apm-agent-python

https://www.elastic.co/guide/en/apm/agent/python/current/index.html
BSD 3-Clause "New" or "Revised" License
413 stars 219 forks source link

psycopg pool not instrumented #2094

Open plexaikm opened 3 months ago

plexaikm commented 3 months ago

We use psycopg connection pool in out FastAPI auto instrumented component, previously we used psycopg.connect directly without connection pool and PG spans were properly reported after switching to connection pool we no longer generated PG spans...

Investigated the auto instrumentation, it binds on psycopg.connect, however the connection pool in pool.py/_connect line 598 method use the Connection class connect method directly bypassing the instrumented instruction...

conn = self.connection_class.connect(self.conninfo, **kwargs)

Local (and quite ugly) fix is to provide ConnectionPool a custom connection_class using the instrumented psycopg.connect method

class WrapperConnectionClass(Connection):

  @classmethod
  def connect(
    cls,
    conninfo: str = "",
    **kwargs: Union[None, int, str],
  ) --> "Connection[Row]":

    return psycopg.connect(conninfo, **kwargs)

...
pool = ConnectionPool(
  ...
  connection_class=WrapperConnectionClass,
  ...
)

This workaround solves the problem locally and we now generate PG spans, but a proper instrumentation solution will be better

To Reproduce

import psycopg
from psycopg_pool import ConnectionPool
import elasticapm

apm_client = elasticapm.Client()
elasticapm.instrument()

pool = ConnectionPool(
  "... conn info ..",
  min_size=1,
  max_size=8,
  check=ConnectionPool.check_connection
)
pool.wait()

with pool.connection() as conn:
  with conn.cursor() as curr:
    curr.execute("SELECT VERSION()")
    curr.fetchone()

Environment (please complete the following information)

Additional context

Add any other context about the problem here.

xrmx commented 3 months ago

Thanks for reporting and for providing a reproducer and even a workaround. Proper instrumentation would be indeed better.