SqliteError: database is locked

Dakad commented 1 week ago

Description

We recently transitioned from Raindrop to Hoarder and using a custom script with the hoarder-cli, we were able to import my Raindrop collections and link them to the matching list on Hoarder but a lot of jobs are stuck in the Queued state.

And checking the docker-compose logs, we can see a lot of crawling errors but the most concerning are these:

hoarder        | 2024-09-10T10:46:46.386869950Z SqliteError: database is locked
hoarder        | 2024-09-10T10:46:46.386927650Z     at w.values (/app/apps/web/.next/server/chunks/917.js:51:42087)
hoarder        | 2024-09-10T10:46:46.386935631Z     ... 3 lines matching cause stack trace ...
hoarder        | 2024-09-10T10:46:46.386941353Z     at m.then (/app/apps/web/.next/server/chunks/917.js:51:45895) {
hoarder        | 2024-09-10T10:46:46.386946297Z   code: 'INTERNAL_SERVER_ERROR',
hoarder        | 2024-09-10T10:46:46.386950667Z   name: 'TRPCError',
hoarder        | 2024-09-10T10:46:46.386960185Z   [cause]: SqliteError: database is locked
hoarder        | 2024-09-10T10:46:46.386964851Z       at w.values (/app/apps/web/.next/server/chunks/917.js:51:42087)
hoarder        | 2024-09-10T10:46:46.386969425Z       at w.all (/app/apps/web/.next/server/chunks/917.js:51:41676)
hoarder        | 2024-09-10T10:46:46.386974091Z       at m.all (/app/apps/web/.next/server/chunks/917.js:51:66818)
hoarder        | 2024-09-10T10:46:46.386978683Z       at m.execute (/app/apps/web/.next/server/chunks/917.js:51:66919)
hoarder        | 2024-09-10T10:46:46.386983405Z       at m.then (/app/apps/web/.next/server/chunks/917.js:51:45895) {
hoarder        | 2024-09-10T10:46:46.386987924Z     code: 'SQLITE_BUSY'
hoarder        | 2024-09-10T10:46:46.386992646Z   }
hoarder        | 2024-09-10T10:46:46.386996942Z }

Details:

Device:

product: Raspberry Pi 5 Model B Rev 1.0
Distributor ID: Debian
Description:    Debian GNU/Linux 12 (bookworm)
Release:        12
Codename:       bookworm

docker-compose.yml

hoarder:
  image: ghcr.io/hoarder-app/hoarder:${HOARDER_VERSION:-release}
  container_name: hoarder
  restart: unless-stopped
  env_file:
    - ~/.env
  environment:
    DATA_DIR: /data
    HOSTNAME: "0.0.0.0"
    NEXTAUTH_SECRET: "${NEXTAUTH_SECRET}"
    MEILI_MASTER_KEY: "${MEILI_MASTER_KEY}"
  volumes:
    - ${CONFIGS_DIR}/hoarder/app:/data
  ports:
    - 3000:3000
  depends_on:
    - chrome
    - meilisearch

.ENV file

HOARDER_VERSION=0.16.0
DISABLE_SIGNUPS=true
# API_URL=http://hoarder:3000
# NEXTAUTH_URL=http://hoarder-app:3000
NEXTAUTH_SECRET="************"
MEILI_ADDR="http://meilisearch:7700"
BROWSER_WEB_URL="http://alpine-chrome:9222"
CRAWLER_NUM_WORKERS=4

OPENAI_API_KEY="sk-*****"
MAX_ASSET_SIZE_MB=30
CRAWLER_STORE_SCREENSHOT=true
CRAWLER_FULL_PAGE_SCREENSHOT=true
CRAWLER_NAVIGATE_TIMEOUT_SEC=90
CRAWLER_FULL_PAGE_ARCHIVE=true

MohamedBassem commented 1 week ago

did this get resolved by any chance or are you still seeing database is locked?

shlajin commented 5 days ago

I'm observing the same problem, but I quickly realized mine is because I'm trying to run SQLite over an SMB share.

MohamedBassem commented 3 days ago

@shlajin for what it's worth. I've been running my own hoarder instance with sqlite over a network share for months now without issues. I'm using NFS though not SMB. But yeah, sqlite recommends running it over network shares so I wouldn't be surprised if that's the issue.

This stackoverflow answer seems to provide some mitigations to fix the locked sqlite problem (https://stackoverflow.com/questions/151026/how-do-i-unlock-a-sqlite-database).

hoarder-app / hoarder

SqliteError: database is locked #385

Description

Details: