search

maxpert / marmot

A distributed SQLite replicator built on top of NATS
https://maxpert.github.io/marmot/
MIT License
1.7k stars 42 forks source link

Unable to scan global changes error no such table __marmot___change_log_global #46

Closed disarticulate closed 1 year ago

disarticulate commented 1 year ago

I tried the binaries at AMD64 0.7.4 and 0.7.5

4:03PM DBG Opening database path=???????????????????????????????????????/scavenger-beta.db
4:03PM DBG Forcing WAL checkpoint
4:03PM DBG Stream ready... leader=lily_nats name=eros-changes-c-1 replicas=1 shard=1
4:03PM INF Listing tables to watch...
4:03PM INF Starting change data capture pipeline...
4:03PM DBG Listening stream shard=1
4:03PM DBG duration=0.394929 name=scan_changes
4:03PM ERR Unable to scan global changes error="no such table: __marmot___change_log_global"

The db isn't open and no other errs are posted.

maxpert commented 1 year ago

I believe your DB is empty with no tables.

disarticulate commented 1 year ago

that might be true. Using docker, mounting things tends to get confusing.

With that said, are you checking whether the passed in database is a accessible file with read/write permissions?

Many sqlite clients seem to greedily create file or memory databases.

maxpert commented 1 year ago

There is no explicit check for read/write but since when Marmot is booting up it will try to create changelog tables and recreate triggers the boot process should fail if we don't get write permissions. Marmot greedily creates file or in memory DB as well, which could be a potential fix.

disarticulate commented 1 year ago

I think you'd create the file. The most interesting/useful scenario i can think of is being able to just push marmot to an endpoint, start it up, then remotely bootstrap it with a schema via some other proces.

The only thing I'd be concerned about is somewhere in the text it mentions that two clients need to have the same db schema, so this would conceivably only work once, then you'd have to manually copy the DB elsewhere.

maxpert commented 1 year ago

If we are missing DB we can absolutely restore a snapshot (and I believe if a snapshot exists, it already restores snapshot.

disarticulate commented 1 year ago

I'm not seeing snapshots being restored atm. I tried the nats and S3 configuration and could verify the S3 stored a snapshot.

I believe this specific error was due to missing DB. A warning log would be useful to debug, and opening a memory DB with a commandline flag to write a file DB would seemingly fix the "Bootstrap" issue. We can close this issue.