Open hrstoyanov opened 5 years ago
Hi Hristo! Thanks for your attention :-) BarbelHisto is meant as a lightweight, easy to understand and use, scalable solution for NoSql Databases. It‘s not meant to compete with Hinernate in the area of OR-Mapping, although you could use the event mechanism to persist the Bitemporal obejcts to any kind of SQL/JDBC datasource. I will think about how it would make sense to support it build-in.
@nschlimm Thanks for the quick response.
I was not focused on Hibernate ORM itself as much as Hibernate/Enver.
As I mentioned, the problem with temporal modeling is much more acute in the world of SQL , just because the NoSQL world is much smaller world (and MongoDB in particular has been in decline for some time now, IMO).
My suggestion is to state upfront and first that BarbelHisto does not work for SQL/JDBC (yet). That would save time to people who look for SQL solution, without having to reading source code and the entire documentation. This was not obvious to me at all (Not that I personally regret reading the docs!)
My experience is that bi-temporal design become very painful when you query temporal entities in relation, say payroll report that links Employee-Rate-Department, where each one is bitemporal. Queries look deceptively simple if you only look at Employee only (for example) . I might have missed that in your test cases (sorry) but that is what I was looking for .
Thanks again!
Hi Hristo ! I will put that NoSql disclaimer upfront in the docs ... I agree ... the complexity that comes with OR-mapping is one reason for me to use NoSQL if possible, especially when bi-temporal is required
BTW: BarbelHisto provides disk persistence and SQL-like indexed quries on the collection ... this pair is blazing fast compared to the assembling required in OR solutions ... on the other hand there are many situation and requirements why people choose a relational database. I will investigate a Hibernate intergration in near future to see if that makes sense ... I have to dive into that
Are you sure Envers provides bi-temporal? My level of knowledge is that envers provides audit trail, i.e. record time, not effective time ...
Thanks @nschlimm
You are correct about Enver not being bi-temporal, just providing auditing.
I am not asking for Hibernate integration, but will welcome any effort in this direction! This tutorial has some valuable ideas/hooks on how to do it with plain JPA. While at that, you may consider integration with JOOQ.
I myself am not big fan of traditional RDBMS either, after working with them for so many years. I am just saying that in Finacial/Insurance corporation bi-temporal design over Oracle or MSSQL is the only design choice you will have. I am speaking from experience here ...
Thanks for your valuable input ! I will definitely look into that soon and comment here when there are any news ...
@hrstoyanov Hi Hristo, I have made it more obvious now right on the landing page, that BarbelHisto (for now) adresses NoSQL data sources. Appreciate your feedback if that is more intuitive now.
@hrstoyanov I noticed that Oracle Database supports JavaScript Object Notation (JSON) data natively with relational database features, including transactions, indexing, declarative querying, and views. I could integrate Oracle easily using that feature. Would that be sifficient or is there any need to dissassemble the object into its relational equivalents?
Hi! Yes, Oracle has some JSON extension, but only in the latest versions(12c+). PostgreSQL also implements binary JSON, arguably better than MongoDB. So PostgreSQL could be a better target ..
@hrstoyanov I could develop an SQL-persistence integration that uses one relation (table). that relation contains the bitemporal stamp fields (version fields) as native SQL fields and the managed pojo (say the a Policy or an Employee or wthatever) as JSON, which then is accessible in the "better" SQL databases like Oracle and PostgreSQL. This kind of solution would work with any SQL database.
Appreciate any of your thoughts ...
@nschlimm Can you pls put something like a design draft, e.g. sq--design.md int he repo with more details?
The implementations I have seen so far are like:
Table A:
- pk
- valid_from: timestamp -- when the record is valid from
- valid_to: timestamp -- when the record is valid to. NULL indicates the latest record
- <other fields > -- other fields that change
The latest record always has valid_to as NULL, any change to the biz fields in the table appends a new record with new <valid_from, NULL> and closes the old most-current with setting timestamp to its "valid_to".
Back-in-time query - need "to" and "from". "to" can be null:
select * from (select * from A where valid_from >= :from and valid_to<= nvl(:to,valid_to) ) -- NVL is a Oracle function
Latest record query:
select * from (select * from A where valid_to is null )
This can be extended to multiple tables:
-- Latest
select * from
(select * from A where valid_to is null ) A join
(select * from B where valid_to is null ) B join
on B.foregn_key_ref=A.pk
-- Back-in-time
select A.*, B.*
from (select * from A where valid_from >= :from and valid_to<= nvl(:to,valid_to) ) A
join (select * from B where valid_from >= :from and valid_to<= nvl(:to,valid_to) ) B
on B.foregn_key_ref=A.pk.
I think Hibernate/Enver takes the design one step further - it places the
@nschlimm Can you pls put something like a design draft, e.g. sq--design.md int he repo with more details?
The implementations I have seen so far are like:
Table A:
- pk
- valid_from: timestamp -- when the record is valid from
- valid_to: timestamp -- when the record is valid to. NULL indicates the latest record
- <other fields > -- other fields that change
The latest record always has valid_to as NULL, any change to the biz fields in the table appends a new record with new <valid_from, NULL> and closes the old most-current with setting timestamp to its "valid_to".
Back-in-time query - need "to" and "from". "to" can be null:
select * from (select * from A where valid_from >= :from and valid_to<= nvl(:to,valid_to) ) -- NVL is a Oracle function
Latest record query:
select * from (select * from A where valid_to is null )
This can be extended to multiple tables:
-- Latest
select * from
(select * from A where valid_to is null ) A join
(select * from B where valid_to is null ) B join
on B.foregn_key_ref=A.pk
-- Back-in-time
select A.*, B.*
from (select * from A where valid_from >= :from and valid_to<= nvl(:to,valid_to) ) A
join (select * from B where valid_from >= :from and valid_to<= nvl(:to,valid_to) ) B
on B.foregn_key_ref=A.pk.
I think Hibernate/Enver takes the design one step further - it places the
hi @hrstoyanov sorry, I was sick the last week. I have looked into the JPA subject more in detail and I do not see a real benefit to use JPA for bi-temporal data. Building a general solution for JPA (i.e. SQL-based DB) would require loads of assumptions being made, not speaking about the mega assembly and disassembly headache that ORM causes in general, and with bi-temporal in particular. I will concentrate on the following options of which I hope that at least one is interesting for you :-)
I don't see much interest in jpa either
.. after reading the docs I realized that BarbelHisto does not work with JDBC (Oracle, MySql, PostreSQL).
Really?
I see there is Mongo engine, but Mongo is a very limited db engine.
The real pain with bi-temporal design is when you have to query a set of related navigable entities, not just a single entity taken (document?) in isolation (which is what i see in your tests cases/docs?). An i don't see this addressed....
For now Hibernate/Enver would still be my preferred technology for dealing with bitemporal data in RDBMS, but I will keep an eye on BabekHisto in case its starts moving in the right direction!
Thanks!