pudo / dataset

Easy-to-use data handling for SQL data stores with support for implicit table creation, bulk loading, and transactions.
https://dataset.readthedocs.org/
MIT License
4.76k stars 297 forks source link

CompileError at `update_many` due to column in a dataset doesn't exist in a db #409

Open 4l1fe opened 1 year ago

4l1fe commented 1 year ago

Got this error while updating a dataset. It's not a big deal and i called the method create_column() to update a table schema,but it's somewhat out of the Dataset concept i guess, columns are created on the go.

As i see, there is the relative issue #352 and the fact that update_many() doesn't handle ensure keyword argument at all.

I think a way to improve the insert_many(), upsert_many(), update_many() methods is to add a new wrapper _sync_columns_many() and move there the block of code:

        # Sync table before inputting rows.
        sync_row = {}
        for row in rows:
            # Only get non-existing columns.
            sync_keys = list(sync_row.keys())
            for key in [k for k in row.keys() if k not in sync_keys]:
                # Get a sample of the new column(s) from the row.
                sync_row[key] = row[key]
        self._sync_columns(sync_row, ensure, types=types)

What do you think about? It doesn't look difficult to fix and i could implement it, gratefully tests are on the place.

czertyaka commented 1 year ago

I've also encountered into this. It seems that MetaData constructor from SQLAlchemy does not support bind keyword argument anymore. main branch:

class MetaData(HasSchemaAttr):
    def __init__(
        self,
        schema: Optional[str] = None,
        quote_schema: Optional[bool] = None,
        naming_convention: Optional[Dict[str, str]] = None,
        info: Optional[_InfoType] = None,
    ) -> None:

It was already marked as deprecated in release 1.4:

class MetaData(SchemaItem):
    @util.deprecated_params(
        bind=(
            "2.0",
            "The :paramref:`_schema.MetaData.bind` argument is deprecated and "
            "will be removed in SQLAlchemy 2.0.",
        ),
    )
    def __init__(
        self,
        bind=None,
        schema=None,
        quote_schema=None,
        naming_convention=None,
        info=None,
    ):
vladiscripts commented 7 months ago

Now the code breaks with a critical error raised:

db['tbl'].all()

TypeError: MetaData.__init__() got an unexpected keyword argument 'bind'