opral / lix-sdk

1 stars 0 forks source link

re-visit simplifying the change queue implementation #35

Open samuelstroschein opened 2 months ago

samuelstroschein commented 2 months ago

Context

The change queue could be implemented without a file_internal table.

Uncertainty exists if the change queue implementation in https://github.com/opral/monorepo/pull/3061 is the "simplest/best" way to go.

Proposal

Decide if the change queue should be implemented with a file_internal table or by storing both old and new file values in the queue itself.

samuelstroschein commented 2 months ago

The file_internal implementations leads to bugs. An argument to go with a copy old, new file data implementation.

The code below has two problems:

  1. I forgot to use settled, expecting that file writes which happen in resolveConflict are immediately written "to disk".
  2. Even using settled didn't work.
CleanShot 2024-08-24 at 17.40.09@2x.png

The fix was to write to the file table instead of file_internal. Point being, the ergonomics around

samuelstroschein commented 2 months ago

Just ran into another problem with file_internal:

SQLite3Error: SQLITE_ERROR: sqlite3 result code 1: cannot UPSERT a view. 

If i am not mistaken, the current queue implementation will prevent upserts. No matter how we design the public API for files. The two options of 1) writing to a file view that triggers the queue, or 2) writing to "the actual table" and bypassing the queue will always lead to a "I want to upsert but don't bypass the queue" problems.

samuelstroschein commented 1 month ago

The flaky tests with 'driver destroyed" are coming from the current change queue implementation https://github.com/opral/monorepo/pull/3093/commits/75fe4be582f839485b2b4f75b05dd617d561f222.