schireson / pytest-mock-resources

Pytest Fixtures that let you actually test against external resource (Postgres, Mongo, Redshift...) dependent code.
https://pytest-mock-resources.readthedocs.io/en/latest/quickstart.html
MIT License
179 stars 19 forks source link

Add in Left/Right UDF's for Redshift #164

Closed jarrettalexander77 closed 2 years ago

jarrettalexander77 commented 2 years ago

Describe the bug Redshift left/right behave differently than the left/right in postgres. Redshift allows integers to be passed in but postgres does not. We have fixed this locally by creating some UDF's in our fixture upon startup. The SQL for these is as follows:

CREATE OR REPLACE FUNCTION left (
    s1 integer, s2 integer
) RETURNS integer
    LANGUAGE sql IMMUTABLE LEAKPROOF AS
'SELECT left(s1::text, s2)::integer';

The UDF for right looks the same.

We would like to contribute this back to the repo so that future users don’t have to deal with the same issue. I found this file here as a starting point but noticed that it seems pretty full of just datediff stuff with some reusable components near the top. I am looking for guidance on where to add this change. I was thinking of splitting out the date diff into its own file and adding this new change as part of another new file in that same folder.

Let me know if you have any other thoughts. I will begin work on this once we have some ideas as to where it will live. I believe that we could address issue #53 as well after we know a better way to layout those UDF's.

DanCardin commented 2 years ago

Hi! Since we've been adding UDFs pretty ad-hoc as we personally encounter usecases that affect us, I'm not entirely sure how large this file could reasonably expect to grow, so i wouldn't be disgruntled if you kept it to the same file.

However, if you were so inclined to refactor them into separate modules, I could imagine it looking something like:

path/
  to/
    udfs/
      __init__.py
      base.py
      dateadd.py
      ...etc

where base.py defines UdfLanguage, and create_udf; the definitions move to their component modules and init.py looks something like:

# __init__.py
from ....udfs.dateadd import dateadd_date, etc...

REDSHIFT_UDFS = Statements(
    dateadd_date,
    ...
)

__all__ = [
    "REDSHIFT_UDFS",
]

This setup being what immediately comes to mind as far as backwards compatibiltiy and avoiding circular imports

DanCardin commented 2 years ago

This is available on main or will be released in the next day or so @ 2.5.1, so i'm gonna close it out. thanks for the contribution!