search

aaronwmorris / indi-allsky

Software to manage a Linux-based All Sky Camera.
GNU General Public License v3.0
252 stars 41 forks source link

Database malformed #1534

Open nicholasmparker opened 1 month ago

nicholasmparker commented 1 month ago

Started getting an issue last night: Error: stepping, database disk image is malformed (11)

I tried vacuum - did not work. Any ideas? I can restore the DB to a backup if needed

aaronwmorris commented 1 month ago

The database might be toast, but try an integrity check.

sqlite3 /var/lib/indi-allsky/indi-allsky.sqlite "PRAGMA integrity_check;"
nicholasmparker commented 1 month ago

It's toast. 🤷

On Tue, Sep 24, 2024, 11:33 AM Aaron W Morris @.***> wrote:

The database might be toast, but try an integrity check.

sqlite3 /var/lib/indi-allsky/indi-allsky.sqlite "PRAGMA integrity_check;"

— Reply to this email directly, view it on GitHub https://github.com/aaronwmorris/indi-allsky/issues/1534#issuecomment-2371780577, or unsubscribe https://github.com/notifications/unsubscribe-auth/AITIPCI5I75MWSMQOS2MUPTZYGH5XAVCNFSM6AAAAABOYWGS6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNZRG44DANJXG4 . You are receiving this because you authored the thread.Message ID: @.***>

aaronwmorris commented 1 month ago

The DB image import is probably not working currently, I need to update some of the patterns.

I have this page for rebuilding the database: https://github.com/aaronwmorris/indi-allsky/wiki/Rebuilding-the-database if you do not have a recent backup.

aaronwmorris commented 1 month ago

I reviewed the DB image import functionality and I think it may work in its current state. Please feel free to try it.

Scratch that. I found something that needed to be updated. Merged #1538

aaronwmorris commented 2 weeks ago

Hopefully, the database rebuilt went smoothly. I am going to close this for now. Please re-open if there is anything else I can do for you.

nicholasmparker commented 2 weeks ago

Thanks Aaron. I rebuilt it successfully. I have noticed - and I havent had enough time to look more closely what is happening, BUT - it seems like if im changing settings and doing a reload that i am more likely to run into the corrupted file issue. Once I settled on settings. for my install, i moved it over to MariaDB and havent had further issues. Have you seen similar?

On Mon, Nov 4, 2024 at 1:41 PM Aaron W Morris @.***> wrote:

Hopefully, the database rebuilt went smoothly. I am going to close this for now. Please re-open if there is anything else I can do for you.

— Reply to this email directly, view it on GitHub https://github.com/aaronwmorris/indi-allsky/issues/1534#issuecomment-2455553593, or unsubscribe https://github.com/notifications/unsubscribe-auth/AITIPCIWYJJ2SIUJGZLSYX3Z665WLAVCNFSM6AAAAABOYWGS6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJVGU2TGNJZGM . You are receiving this because you authored the thread.Message ID: @.***>

aaronwmorris commented 2 weeks ago

There were some issues at one point when I had the sqlite database pragma set to disable synchronous mode in a misguided attempt to resolve locked database problems. Setting synchronous=off made things faster, but it was very prone to database corruption if there is a power loss.

This has not been the case for a while and the standard setting now is synchronous=normal. Database corruption has been a very rare issue.

https://www.sqlite.org/pragma.html#pragma_synchronous

nicholasmparker commented 2 weeks ago

Got it - i will let you know on this thread if it happens again. Im setting up an imx462 version for my brother, so will tinker with it on sqlite a bit to see if I can reproduce it.

On Mon, Nov 4, 2024 at 6:08 PM Aaron W Morris @.***> wrote:

There were some issues at one point when I had the sqlite database pragma set to disable synchronous mode in a misguided attempt to resolve locked database problems. Setting synchronous=off made things faster, but it was very prone to database corruption if there is a power loss.

This has not been the case for a while and the standard setting now is synchronous=normal. Database corruption has been a very rare issue.

https://www.sqlite.org/pragma.html#pragma_synchronous

— Reply to this email directly, view it on GitHub https://github.com/aaronwmorris/indi-allsky/issues/1534#issuecomment-2455955416, or unsubscribe https://github.com/notifications/unsubscribe-auth/AITIPCJUFHVTNMSWXJ6BNMDZ67455AVCNFSM6AAAAABOYWGS6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJVHE2TKNBRGY . You are receiving this because you authored the thread.Message ID: @.***>

nicholasmparker commented 1 week ago

Happened again. I've tried two different SD cards too, so likely not the issue. I wonder if its an issue with the power from the raspberry POE hat or maybe a defective raspi

On Mon, Nov 4, 2024 at 7:11 PM Nicholas Parker @.***> wrote:

Got it - i will let you know on this thread if it happens again. Im setting up an imx462 version for my brother, so will tinker with it on sqlite a bit to see if I can reproduce it.

On Mon, Nov 4, 2024 at 6:08 PM Aaron W Morris @.***> wrote:

There were some issues at one point when I had the sqlite database pragma set to disable synchronous mode in a misguided attempt to resolve locked database problems. Setting synchronous=off made things faster, but it was very prone to database corruption if there is a power loss.

This has not been the case for a while and the standard setting now is synchronous=normal. Database corruption has been a very rare issue.

https://www.sqlite.org/pragma.html#pragma_synchronous

— Reply to this email directly, view it on GitHub https://github.com/aaronwmorris/indi-allsky/issues/1534#issuecomment-2455955416, or unsubscribe https://github.com/notifications/unsubscribe-auth/AITIPCJUFHVTNMSWXJ6BNMDZ67455AVCNFSM6AAAAABOYWGS6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJVHE2TKNBRGY . You are receiving this because you authored the thread.Message ID: @.***>

aaronwmorris commented 1 week ago

That is very strange. Did you have any strange error messages in the system log?