search

aergoio / litetree

SQLite with Branches
MIT License
1.62k stars 35 forks source link

Document how to merge newer SQLite releases #18

Open snej opened 5 years ago

snej commented 5 years ago

As a fork, LiteTree is inevitably going to need updating as newer versions of SQLite are released (At the moment SQLite is up to 3.30, while litetree is still based on 3.27.2.) Keeping up with SQLite can be very important if one needs new features or bug fixes.

How does one perform such an update? All I see in here are heavily modified versions of sqlite3.c and sqlite3.h, not any tools for updating the source. Does one just run a 3-way-merge tool and feed it litetree's source, the unmodified version of the same SQLite release, and the new SQLite release?

It would be good to document this, so potential users of the library can feel that they're not going to be stuck on an old release of SQLite forever, or be dependent on someone else to keep it up to date for them.

snej commented 5 years ago

I diffed sqlite3.c against the stock 3.27.2 version. It looks like you've been developing with individual files but only committing the amalgamation here — I see auto-generated comments like --- begin file branches.c ---. It would be very helpful if this repo contained the actual files you're working on, not just the amalgamation.

kroggen commented 5 years ago

Hi!

I edit the amalgamation file directly. That comment is just to follow the rules.

The update will be done on next month (I am busy now) probably against SQLite 3.31 (with virtual/generated columns). Documentation can be made at the same time.

aspiers commented 4 years ago

This might be a crazy idea - but what are the chances of getting the litetree functionality actually merged into SQLite core, e.g. as an optional extra feature which could be enabled at compile-time or even run-time? If that was possible, this problem would vanish :-)

kroggen commented 4 years ago

Yes, this idea was considered when it was developed.

But there are at least 2 items that make it unpractical:

  1. The format of the pragma commands used by LiteTree is not the default one of SQLite, so it requires to pre-parse these commands before SQLite engine does that.

  2. SQLite does not expect a change in schema and it does not reload the cache on normal commands. LiteTree does a required reload of all of these when a command to change to another branch is executed.

I have not analyzed the source now but it may have other reasons too.

aspiers commented 4 years ago

Thanks for the reply! Couldn't those differences be encapsulated within #ifdef LITETREE blocks or similar?

I guess the other option is to standardise on a workflow based on git merge or rebase, but that sounds to me like it would probably be a lot more painful in the long run.

kroggen commented 4 years ago

Sometimes a new version of SQLite comes with changes exactly where the patches would be applied or even on other places that affect the functionality.

Running the functional tests help to discover if something was broken, but it is not guaranteed that the tests will cover all cases. This is why I make a manual review of the changes.