search

AugurProject / augur-app

Augur Desktop Electron Application
MIT License
222 stars 22 forks source link

[BUG] Intermittent SQLIte3 connection errors prevent syncing until app is restarted #442

Closed ndorf closed 5 years ago

ndorf commented 5 years ago

Summary

The Augur app sometimes gets stuck on SQLite3 connection errors, which persist and block any syncing progress until the app is restarted. Everything seems to work fine after the restart.

Steps to reproduce

Not sure, it seems to happen more or less randomly. I have seen it occur after the app has been up and syncing normally for a while, as well as shortly after starting and before fully syncing in the first place.

It's happened for me twice in the last few days with "Local Light Node (experimental)." I think I've also seen it in the past using Alchemy and even gethnode (back when that worked), but can't prove it yet.

Expected behavior

No errors, or at least no need to restart the app manually.

Screenshots

Terminal output from augur-app:

[14:39:11.431] [info] AugurNodeServer requestLatestSyncedBlock
[14:39:15.839] [info] {"id":1760,"jsonrpc":"2.0","method":"getSyncData","params":null}
[14:39:15.839] [info] {"id":1761,"jsonrpc":"2.0","method":"getSyncData","params":null}
[14:39:15.841] [error] getter error:  Error: Unable to acquire a connection
    at Client_SQLite3.acquireConnection (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/client.js:318:40)
    at Runner.ensureConnection (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/runner.js:227:24)
    at Runner.run (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/runner.js:47:42)
    at Builder.Target.then (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/interface.js:35:43)
    at process._tickCallback (internal/process/next_tick.js:188:7)

[14:39:15.841] [error] getter error:  Error: Unable to acquire a connection
    at Client_SQLite3.acquireConnection (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/client.js:318:40)
    at Runner.ensureConnection (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/runner.js:227:24)
    at Runner.run (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/runner.js:47:42)
    at Builder.Target.then (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/interface.js:35:43)
    at process._tickCallback (internal/process/next_tick.js:188:7)

[14:39:16.279] [info] {"id":1762,"jsonrpc":"2.0","method":"getMarketPriceCandlesticks","params":{"marketId":"0x8c8215d01e9e735e2522e44cd4e8e8fec06a0437","period":86400,"start":1546041600,"end":1546724322,"outcome":"1"}}
[14:39:16.279] [error] getter error:  Error: Unable to acquire a connection
    at Client_SQLite3.acquireConnection (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/client.js:318:40)
    at Runner.ensureConnection (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/runner.js:227:24)
    at Runner.run (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/runner.js:47:42)
    at Builder.Target.then (/usr/local/src/upstream/augur-app/dist/linux-unpacked/resources/app.asar/node_modules/augur-node/node_modules/knex/lib/interface.js:35:43)
    at process._tickCallback (internal/process/next_tick.js:188:7)

Environment

adrake33 commented 5 years ago

This might have been fixed by the recent changes made to Augur Node, but it will need more investigation by one of the devs to confirm.

adrake33 commented 5 years ago

@ndorf This should be fixed in version 1.9.0 of Augur App. Can you please upgrade to this version and let us know if you are still seeing this issue?

adrake33 commented 5 years ago

@ndorf Sorry, I had meant to say that #439 should be fixed now. I'm actually not sure if this issue has been resolved yet. Can you please give it a try with the latest version and let us know if it's still happening for you?

ndorf commented 5 years ago

@adrake33 Sure. The previous version ran fine for 10 straight days before I finally saw this error again yesterday. So, it won't be immediately obvious, but I'll update this if/when it does happen.

adrake33 commented 5 years ago

@ndorf Ok, thanks for the update. We will look into this some more.

ndorf commented 5 years ago

@adrake33 Just in case that wasn't 100% clear, I haven't seen the error with the 1.9.1 (having only upgraded to that today). What I meant to say above was simply that even the old, "known-bad" version only exhibits the error on rare occasion, so it will take a while to determine whether it's still an issue with the new one.

In other words, if you have reason to believe this should be fixed, you probably don't need to do anything but wait.

adrake33 commented 5 years ago

@ndorf I don't believe there were any recent changes that would have fixed it, so we'll investigate it more. If you do see it in 1.9.1 or 1.9.2, please let us know just to confirm.

ndorf commented 5 years ago

Got it again with 1.9.1, and with Mainnet (Alchemy) this time.

Is there a way I can increase the length of time past logs are kept for? By the time I noticed this, it was a couple of hours after it occurred and both my current and old log files contained nothing but these messages. It might be nice to see if anything interesting happened right before, but it seems that those logs got purged rather quickly?

adrake33 commented 5 years ago

@ndorf According to https://github.com/megahertz/electron-log#file-transport, there is a maxSize parameter that can be specified in the file transport that sets the maximum size of the file. Setting it to 0 disables this limit. However, once the file reaches the maximum, the older logs get moved to the file log.old.log in the same directory. Can you check that file to see if it provides any more insight?

ndorf commented 5 years ago

Sorry for the lack of update, but the problem hasn't recurred again so far. I did increase the max size as you suggested though, so if/when it does, hopefully I'll be able to post something useful. (I simply added log.transports.file.maxSize=134217728 to the top of src/main/main.js -- not sure if there's a better way?)

As for the log.old.log file, I did check that one last time, but it didn't go far back enough to capture the start of the trouble, unfortunately (2MB goes by pretty fast when there is a stack trace every couple of seconds).

adrake33 commented 5 years ago

Thanks for letting us know, @ndorf! Yes, I believe that is the best way to change the file size in Augur.

Since this hasn't occurred recently, I'll close this issue for now. Feel free to reopen it, though, if you see it happen again.