zodb / relstorage

A backend for ZODB that stores pickles in a relational database.
Other
54 stars 46 forks source link

pg8000 1.17 is incompatible #438

Closed jamadden closed 3 years ago

jamadden commented 3 years ago

It was refactored and some of the details we relied on have changed.

Specifically, the way the Cursor generates rows:

      cursor = self.cursor(conn)
    File "//relstorage/adapters/drivers.py", line 211, in cursor
      cur = conn.cursor()
    File "//relstorage/adapters/postgresql/drivers/pg8000.py", line 156, in cursor
      return Cursor(self)
    File "//relstorage/adapters/postgresql/drivers/pg8000.py", line 95, in __init__
      assert isinstance(self._cached_rows, deque)
  AttributeError: 'Cursor' object has no attribute '_cached_rows'
jamadden commented 3 years ago

There's another issue:

When you execute SQL involving an array parameter, pg8000 now tries to figure out the type of the elements in the array based on the first element. For integers, it examines the numeric value to pick a type:


min_int2, max_int2 = -(2 ** 15), 2 ** 15
min_int4, max_int4 = -(2 ** 31), 2 ** 31
min_int8, max_int8 = -(2 ** 63), 2 ** 63

def inspect_int(value):
    if min_int2 < value < max_int2:
        return PY_TYPES[SMALLINT]
    if min_int4 < value < max_int4:
        return PY_TYPES[INTEGER]
    if min_int8 < value < max_int8:
        return PY_TYPES[BIGINT]
    return PY_TYPES[Decimal]

But if your array has wildly different numeric values, this can fail.

For example, executing DELETE FROM object_ref WHERE tid = ANY (%s) with the parameter ([0, 279098040260880896, 279098040260852974, 279098040260818483],) fails because pg8000 looks at the 0 and guesses a SMALLINT, but that doesn't work for the second and subsequent values:

pg8000.dbapi.ProgrammingError: {'S': 'ERROR', 
   'V': 'ERROR', 
   'C': '22003', 
   'M': 'value "279098040260880896" is out of range for type smallint', 
   'F': 'numutils.c', 'L': '241', 'R': 'pg_strtoint16'}