kvesteri / sqlalchemy-continuum

Versioning extension for SQLAlchemy.
BSD 3-Clause "New" or "Revised" License
582 stars 125 forks source link

Empty changesets / versions created when using high precision floats #219

Open rosensama opened 5 years ago

rosensama commented 5 years ago

I'm trying to write an idempotent migration script and avoid updates when column values don't change. I can see that sqlalchemy-continuum usually does this, but I've found a case where different float representations seems to break that and I get unnecessary, empty record revisions.

Is there a recommended pattern for dealing with that? Details follow.

I added logging every time is_modified says an object has changed to see what's triggering the empty changesets.

    records = session.query(MyModel)
    for record in records:
        prior_value = record_value
        record.value = value
        if session.is_modified(record):
            LOGGER.info("prior value: %.16f value: %.16f - %d %s", 
                        prior_value, value, record.id, changeset(record))

I see this on every run where the changeset winds up empty, so it seems SQLA sees it as a change, which triggers continuum. But then later on continuum sees no change set and PropertyModTrackerPlugin doesn't flag value as modified.

prior value: -35.4662261975366988 value: -35.4662261975366562 - 1466 {'value': [-35.466226197536656, -35.4662261975367]}

A check for how many revisions were made to the record:

record = session.query(MyModel).filter_by(id=1466).one()
for version in record.versions:
    print(version.changeset)

Output:

{'id': [None, 1466]}
{'value': [None, -35.4662261975367]}
{}
{}
{}

Sorry, but I cannot find a simple repro for this. It's in a script copying data from two different database engines (sqlite -> postgres). The source is a REAL in sqlite read through SQLA inspection, Table(autoload=True). The target is a double precision in Postgres defined in SQLA as a Float.