cal-itp / data-infra

Cal-ITP data infrastructure
https://docs.calitp.org/data-infra
GNU Affero General Public License v3.0
44 stars 12 forks source link

Future sqlalchemy issues with calitp-data-analysis package #3361

Open vevetron opened 1 month ago

vevetron commented 1 month ago

User story / feature request

When trying to update this pr, failed to update sqlalchemy packages since after 2.0 they made breaking changes.

Will probably need to update at least the sql file.

You can see some of the error messages on this run: https://github.com/cal-itp/data-infra/actions/runs/9218744599/job/25362779898?pr=3357 Some of the errors: `==================================== ERRORS ==================================== __ ERROR at teardown of test_auto_table_write __

@pytest.fixture
def tmp_name():
    from sqlalchemy.exc import NoSuchTableError

    # Code that runs before test ----
    # generate a random table name. ensure it does not start with a number.
    table_name = "t_" + str(uuid.uuid4()).replace("-", "_")
    schema_table = f"***CI_SCHEMA_NAME***.***table_name***"

    # pass the name of the temporary table into the test
    yield schema_table

    # Code that runs after test ----
    # delete table corresponding to temporary name, if it exists
    try:
      tbl = get_table(schema_table)

tests/conftest.py:29:


table_name = 'calitp_py.t_4dc1294d_da8f_4853_a1ea_c543ceaf1db6', as_df = False

def get_table(table_name, as_df=False):
    engine = get_engine()
    src_table = format_table_name(table_name)
  table = Table(src_table, MetaData(bind=engine), autoload=True)

E TypeError: MetaData.init() got an unexpected keyword argument 'bind'

calitp_data_analysis/sql.py:82: TypeError`

Please describe your need, outlining the key users, the feature being requested, and the goal that that the feature will facilitate. For example: As a [user or stakeholder type], I want [software feature] so that [some business value]

Acceptance Criteria

Able to use sqlalchemy 2.0.3 in the calitp-data-analysis package

Profit!

vevetron commented 1 month ago

This might not be doable since https://github.com/machow/siuba 0.4.4 requires SQLAlchemy==1.3.18. Depends on how dependencies work - do all of them need to work together? Or will Siuba's dependences be custom in itself, and won't clash with the other package dependencies.