Closed kentcdodds closed 9 months ago
Does the deploy work if you disable the proxy from litefs.yml
? You may need to temporarily disable the health check as well.
Well, I have good news and bad news. The bad news is that disabling the proxy didn't seem to fix anything. The good news is that this is probably not the fault of LiteFS's proxy and it's my problem not yours 😅
Good grief, when I decided to overscope my personal website to match the constraints, requirements, and use cases of bigger companies with teams of full time developers, I didn't consider the fact that it would take more maintenance than a single person would devote to their side project 🤡 Why do I do this to myself?
That said, can you explain why the proxy server isn't listed in the available ports? Does the proxy server wait for the internal server to start up before it starts?
That said, can you explain why the proxy server isn't listed in the available ports? Does the proxy server wait for the internal server to start up before it starts?
The proxy server doesn't start up until the LiteFS node has sync'd with the primary or has become primary so it may be delayed when starting up.
Ah, and it doesn't become primary until the exec is successful?
Ok, so I shuffled some things around and I'm right back where I started except now it's working 🤷♂️
Feel free to watch in case it's helpful: https://www.loom.com/share/87c43ce85c8a4d58a76d0963b38eebe5
I'm not sure whether this is related, but this started happening yesterday as well, so maybe I'll start having deployment issues again? https://www.loom.com/share/462a794cbd4444928d0c028fb4558ddd
I'm going to scale down to a single region again and see whether that fixes anything.
To circle back, scaling to 1 and restarting the server fixed the last issue. I've scaled up to one additional region and I've enabled tracing (https://fly.io/docs/litefs/config/#trace-log) and we'll keep an eye on it. @benbjohnson thinks this is either LiteFS or my application code putting a lock on the db, though the "SQLite error or database missing" error is strange.
I tried deploying today and got an error. I upgraded to the latest version and received the same error. It appears to be due to the proxy server not starting up. Here are relevant logs from my server during the startup:
That's not entirely helpful. Here are the deploy logs:
That's a bit more helpful. From that it appears that no server is listening on port 8080 as configured in my fly.toml, which is set as the
INTERNAL_PORT
environment variable in my Dockerfile which is referenced as theproxy.addr
in mylitefs.yml
.I'm not sure what's changed from the last successful deploy. I haven't changed anything with LiteFS, Dockerfile, or fly.toml configuration (other than try to upgrade litefs after the first failure).
It does look like
litefs mount
has mounted to port 20202, but I believe that's for the consul service. I think we should see another entry in the processes forlitefs mount
listening on port 8080, and that's why I think this is a litefs issue.