search

pydiverse / pydiverse.pipedag

A data pipeline orchestration library for rapid iterative development with automatic cache invalidation allowing users to focus writing their tasks in pandas, polars, sqlalchemy, ibis, and alike.
https://pydiversepipedag.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
15 stars 2 forks source link

version 0.6.2 writes pandas tables with lowercase names to DB2 #106

Closed windiana42 closed 1 year ago

windiana42 commented 1 year ago

Even if writing table "_FILTER_IDS" as an output of a task with dag.Table(df, "_FILTER_IDS"), the table is written with lowercase name to DB2 even quoted as can be seen in ALTER TABLE statement adding indexes <schema>."_filter_ids".

DB2 has by default uppercase name. So the opposite should actually happen.

NMAC427 commented 1 year ago

Pipedag table and blob names are case insensitive: https://github.com/pydiverse/pydiverse.pipedag/blob/43f0e3bab7e6b16d6f2023d2c87cf1f8c1f34c9a/src/pydiverse/pipedag/util/naming.py#L6-L13

NMAC427 commented 1 year ago

However, this needs to be documenter better.