Support weaker consistency models

[kentcdodds]

I believe there's an issue open about this somewhere, but I can't find it.

Here's my use case:

I'm using SQLite to cache a lot of things in my app like third party APIs and slow queries. Here are two conflicting problems:

1. I can't use LiteFS for this cache because I write to this cache during GET requests all the time and I don't want to replay requests to the primary to update the cache. It would undo the benefits of multi-region because almost every request would be replayed.
2. The caches can (and do) diverge. So a user can hit refresh and see different data because the second cache has stale data that hasn't been refreshed yet. On top of that, when I want to trigger a cache revalidation explicitly, I will have to trigger it across all regions which is a big pain.

Write speed is not important for this cache (I can fire-and-forget), so I'd like to be able to support writes from any region for this specific database. In my use case I'll need this support to not be at the global litefs level, but at the specific database level. This is because I also have another sqlite database that serves as the actual data store for my app and write speed matters for that.

Let me know if you have questions or if I'm missing something. Thanks!

[benbjohnson]

I think the issue you might be referring to is Write Forwarding (https://github.com/superfly/litefs/issues/56), however, that has a lot of latency when you try to do it from distant regions. LiteFS is built around a strong consistency model (serializable) so implementing multi-master isn't currently something it can do.

I'm not opposed to allowing for weaker consistency models but I'd just have to figure out how to do it cleanly. Tools like Mycelial are built around these kinds of models. There's also the session extension which can be used to ship logical changes.

I'll take a look and see if we could integrate something like the session extension into LiteFS for cache data.

[kentcdodds]

that has a lot of latency when you try to do it from distant regions

That's actually fine for me because I'd just fire-and-forget in this case (it's a cache after all).

[kentcdodds]

Again, I want to call out that I would need this to work on a per-database basis. I have one database for my app data and a separate one for the cache and I want to keep the existing behavior for my app data, but have this new behavior for the cache.

[benbjohnson]

that has a lot of latency when you try to do it from distant regions

That's actually fine for me because I'd just fire-and-forget in this case (it's a cache after all).

Another more important downside is that SQLite is single writer so the write lock has to be held for the length of at least 1 round trip. If you have your primary in Denver and you write from Sydney then you might have 250ms of time where the database is locked for writes. That would limit you to 4 writes per second.

I want to call out that I would need this to work on a per-database basis

Good call out. The databases in LiteFS are pretty isolated so most features like this will be on a per-database level.

[kentcdodds]

If you have your primary in Denver and you write from Sydney then you might have 250ms of time where the database is locked for writes. That would limit you to 4 writes per second.

To be clear, the Sydney cache database would be the one that's locked right? The Denver cache wouldn't be locked right? And just because the database is locked for writes doesn't impact reads right? And if another write comes in, it'll just wait, or will it error out?

It's possible that I should just make an internal API call to trigger a cache update that one region can send to the primary so cache writes only happen in the primary, but don't have to involve a fly-replay... 🤔

[benbjohnson]

To be clear, the Sydney cache database would be the one that's locked right? The Denver cache wouldn't be locked right? And just because the database is locked for writes doesn't impact reads right? And if another write comes in, it'll just wait, or will it error out?

The write lock exists on the primary but with Write Forwarding the replicas can borrow the lock temporarily. It's still global so all copies of the database would be locked for writes. Write locks don't affect reads so long as you're using the WAL journaling mode. Writes will get queued if they can't acquire the write lock immediately (and eventually timeout).

It's possible that I should just make an internal API call to trigger a cache update that one region can send to the primary so cache writes only happen in the primary, but don't have to involve a fly-replay... 🤔

This is probably the best approach and it'll work right now. With this approach, the write lock is only acquired for a few milliseconds since all the processing occurs on the primary node.

[kentcdodds]

Makes sense. Yeah, the internal API call sounds the most sensible.

[kentcdodds]

Here's my implementation of that:

https://github.com/kentcdodds/kentcdodds.com/blob/ec9c0d5a652d97c97ee98ffe0d33e38c592ce962/app/routes/resources/cache.sqlite.ts

And here's how I handle when cache.set and cache.delete is called: https://github.com/kentcdodds/kentcdodds.com/blob/ec9c0d5a652d97c97ee98ffe0d33e38c592ce962/app/utils/cache.server.ts#L73-L132

[kentcdodds]

I'm getting this:

2023-01-24T05:15:47Z app[2862cf95] ams [info]FetchError: request to https://3f8f0b85.vm.kcd.internal/resources/cache/sqlite failed, reason: connect ECONNREFUSED fdaa:0:23df:a7b:d826:3:5d51:2:443

What am I doing wrong? I'm guessing my URL is incorrect or something?

[benbjohnson]

If you’re hitting the “internal” DNS then you should use http instead of https. You’ll also need to use the internal port that your server running on (if it’s not using port 80).

[kentcdodds]

That worked 👍 I think this is the best approach. I'm still able to avoid using (or hosting) another service, and the cache is at the edge next to my app 🔥