Open djlambert opened 6 years ago
Did some digging through the tests and added the following code before creating all the tables:
orm.configure_mappers()
versioning_manager.transaction_cls.__table__.create(engine)
versioning_manager.activity_cls.__table__.create(engine)
This seems to have fixed the issue. Is this something missing from the documentation?
I had the same issue, and this solved my problem, too. Base.metadata.create_all(engine) tries to use the audit_table function before it's created, but this is an easy workaround.
Thanks for commenting with your solution, @djlambert - you saved me on this one.
Hey @djlambert and @kevindalias !
Have you had the chance to compare the performance against continuum? Is it really better?
Currently I'm using continuum in Postgresql without native_versioning
, and working towards implementing it (but there's several bugs stopping me, like https://github.com/kvesteri/sqlalchemy-continuum/issues/117 ).
Also, did you manage to migrate all the data to the new unique table in postgresql-audit? Is it even possible?
Thank you!
@rogerio-geru I've used continuum extensively in the past, and I can confidently tell you postgresql-audit is dramatically faster. There's just no way for Python to compete with native Postgres triggers. If speed is a problem for you, the switch is absolutely worth it. Theoretically native-versioning
in continuum would offer a similar benefit, but I never got that working for the same reasons you mentioned.
I didn't migrate the old data from continuum, but I definitely think it's possible. It'd just take a hell of a lot of careful work and testing to get it right. So it lands in that gray area of cost/benefit. In my case, dropping the old data was actually a benefit because it'd become so enormous and unwieldy with continuum.
I've also noticed a number of nice side effects of switching, especially around type handling. I used to have problems with continuum confusing actual changes and type changes (e.g., it was picking up date changes because some modules were sending '2019-01-01' strings instead of datetime objects). Similar issues were occurring with boolean columns (False vs '0' vs 0 in Python). That went away with postgresql-audit since Postgres is in charge.
I know I gave a lot of non-answers, but I hope this helps you make a good decision.
Thank you for your comments @kevindalias! I think I will try it, and leaving behind the history for now, like you did.
But for the moment, I'm struggling a little. Where is the documentation @kvesteri? Isn't there any? I've studied the code, it's very nice, but what's the difference between statement and row level for instance? There isn't even docstrings to explain several parts.
I'll try it next week, but I'm apprehensive atm.
The difference if the type of triggers used for storing changes. For the majority of cases statement level triggers are by far faster than row level triggers.
Statement level trigger for an UPDATE that updates 1000 rows makes only a single version history INSERT whereas row level trigger would issue 1000 inserts.
Thank you @kvesteri, I see.
But what's the format of this statement level Activity, what would the changed_data
be?
Another, more important thing, you have changed the interface from Continuum, for example models do not get a .versions
method anymore, and each activity, which is a version instance, do not have a .changeset
method, which lists changed_data as
{
'name': ['old', 'new']
}
And my code needs these methods. Any advice on how could I reintroduce them? Will I have to fork your repo and implement them in mine?
Well. I've just stumbled upon a very big problem... I don't know if I'm missing something, please help @kvesteri or @kevindalias !
All of my company models use multi-table inheritance in sqlalchemy, so for example I have this:
Base = declarative_base()
class Part(Base):
__tablename__ = 'part'
class Person(Part):
__tablename__ = 'person'
class Profile(Person):
__tablename__ = 'profile'
class Borrower(Profile):
__tablename__ = 'borrower'
When I retrieve my first versions, it'll return all of them apart:
In [7]: from postgresql_audit import versioning_manager
In [8]: Activity = versioning_manager.activity_cls
In [9]: Activity.query.all()
Out[9]:
[<Activity table_name=u'user' id=1L>,
<Activity table_name=u'account' id=2L>,
<Activity table_name=u'part' id=3L>,
<Activity table_name=u'person' id=4L>,
<Activity table_name=u'profile' id=5L>,
<Activity table_name=u'borrower' id=6L>]
How would I retrieve the actual changed data of the Borrower
in question? Doesn't postgresql-audit merge the involved activities in all tables?
It's hard to be sure without more context, but I did notice one particularly important omission in the snippets above: versioning_manager.init(Base)
There's more info on the SQLAlchemy integration here: https://postgresql-audit.readthedocs.io/en/stable/sqlalchemy.html - I found it pretty useful as a basic reference.
I'm trying to replace SQLAlchemy-Continuum in my project with postgresql-audit but must be missing something.
When I replace the call to
make_versioned
withversioning_manager.init(Base)
and try and create the schema it tries to use theaudit_table
function before it's defined.I decided to try something simple to test the functionality, but no activity records are saved (similar to #26)
I'd really like to get this working so I can test the performance against SQLAlchemy-Continuum before putting my project into production. I presume it'll be difficult to migrate after the fact.