hay-kot / homebox

Homebox is the inventory and organization system built for the Home User
https://hay-kot.github.io/homebox/
GNU Affero General Public License v3.0
2.58k stars 176 forks source link

Import "database is locked (5) (SQLITE_BUSY)" errors #906

Open rpavlik opened 1 month ago

rpavlik commented 1 month ago

First Check

Homebox Version

0.10.3

What is the issue you are experiencing?

When importing a CSV file with about 40-50 items, the whole file gets parsed, but the actual import is often terminated prematurely by a "database is locked (5) (SQLITE_BUSY)" error. If every line has an import_ref it's OK to try to import repeatedly, and eventually you'll get most if not all of it imported.

homebox    | 9:48PM DBG go/src/app/internal/core/services/reporting/io_sheet.go:125 > parsed value tag=HB.model_number type=string val=
homebox    | 9:48PM DBG go/src/app/internal/core/services/reporting/io_sheet.go:125 > parsed value tag=HB.serial_number type=string val=
homebox    | 9:48PM DBG go/src/app/internal/core/services/reporting/io_sheet.go:125 > parsed value tag=HB.lifetime_warranty type=bool val=false
homebox    | 9:48PM DBG go/src/app/internal/core/services/reporting/io_sheet.go:125 > parsed value tag=HB.warranty_expires type=types.Date val=
homebox    | 9:48PM DBG go/src/app/internal/core/services/reporting/io_sheet.go:125 > parsed value tag=HB.warranty_details type=string val=
homebox    | 9:48PM DBG go/src/app/internal/core/services/reporting/io_sheet.go:125 > parsed value tag=HB.sold_to type=string val=
homebox    | 9:48PM DBG go/src/app/internal/core/services/reporting/io_sheet.go:125 > parsed value tag=HB.sold_price type=float64 val=0
homebox    | 9:48PM DBG go/src/app/internal/core/services/reporting/io_sheet.go:125 > parsed value tag=HB.sold_time type=types.Date val=
homebox    | 9:48PM DBG go/src/app/internal/core/services/reporting/io_sheet.go:125 > parsed value tag=HB.sold_notes type=string val=
homebox    | 9:48PM INF go/src/app/internal/web/mid/logger.go:36 > request received method=GET path=/api/v1/locations rid=dfe4f17e0746/WDBFajZi5D-000089
homebox    | 9:48PM INF go/src/app/internal/web/mid/logger.go:36 > request received method=GET path=/api/v1/locations rid=dfe4f17e0746/WDBFajZi5D-000090
homebox    | 9:48PM INF go/src/app/internal/web/mid/logger.go:41 > request finished method=GET path=/api/v1/locations rid=dfe4f17e0746/WDBFajZi5D-000089 status=200
homebox    | 9:48PM ERR go/src/app/app/api/handlers/v1/v1_ctrl_items.go:265 > failed to import items error="database is locked (5) (SQLITE_BUSY)"
homebox    | 9:48PM ERR go/src/app/internal/web/mid/errors.go:31 > ERROR occurred error="database is locked (5) (SQLITE_BUSY)" req_id=dfe4f17e0746/WDBFajZi5D-000088

How can the maintainer reproduce the issue?

Generate an inventory CSV with about 40-50 items. (Sorry I cannot share mine, it's actually based on an export of my work snipe-it.) Import while looking at the logs. On the browser you'll get an "import failed" as well as "import success" message, while in the logs you'll see some number of SQLITE_BUSY errors.

Deployment

Docker (Linux)

Deployment Details

Rootless container, directory for the /data volume. Using basically the suggested docker-compose file.

hay-kot commented 1 month ago

Looks like there's a wrong env set for the rootless image. Add the following ENV to your docker-compose and it should resolve the issue. I'll get this fixed in the next release.

HBOX_STORAGE_SQLITE_URL=/data/homebox.db?_pragma=busy_timeout=2000&_pragma=journal_mode=WAL&_fk=1

rpavlik commented 1 month ago

That seems to have helped, thanks! (I still got a "failed to create location" error but I think that is unrelated...)