Persistent lock on the database. How to fix?

[nitobuendia]

Issue

When trying to run gphotos-sync, it stops because the sqllite database is locked.

Running

start_date=$(date +"%F" --date="3 month ago")
docker run --name gphotos --rm -v /mnt/ssd/docker/gphotos/config:/config -v /mnt/nas/Google/Photos:/storage -p 9580:8080 ghcr.io/gilesknap/gphotos-sync:3.1.0 /storage --progress --use-hardlinks --start-date ${start_date}

I normally use docker-compose up, but I was experimenting with a few links.

Logs

[SSH] executing...
03-19 04:33:17 WARNING  gphotos-sync 3.1.0 2023-03-19 04:33:17.677703 
03-19 04:33:18 WARNING  Indexing Google Photos Files ... 
03-19 04:33:18 WARNING  Listed 10 items ... 
03-19 04:33:18 WARNING  Listed 20 items ... 
03-19 04:33:18 WARNING  Listed 30 items ... 
03-19 04:33:19 WARNING  Listed 40 items ... 
03-19 04:33:19 WARNING  Listed 50 items ... 
03-19 04:33:19 WARNING  Listed 60 items ... 
(... quite a few items ...)
03-19 04:33:46 WARNING  Downloading Photos ... 
03-19 04:33:46 WARNING  Downloaded 0 Items, Failed 0, Already Downloaded 13069 
03-19 04:33:51 ERROR    
Process failed. 
Traceback (most recent call last):
  File "/root/.local/lib/python3.10/site-packages/gphotos_sync/Main.py", line 507, in main
    self.start(args)
  File "/root/.local/lib/python3.10/site-packages/gphotos_sync/Main.py", line 447, in start
    self.do_sync(args)
  File "/root/.local/lib/python3.10/site-packages/gphotos_sync/Main.py", line 408, in do_sync
    with self.data_store:
  File "/root/.local/lib/python3.10/site-packages/gphotos_sync/LocalData.py", line 74, in __exit__
    self.store()
  File "/root/.local/lib/python3.10/site-packages/gphotos_sync/LocalData.py", line 79, in store
    self.con.commit()
sqlite3.OperationalError: database is locked
03-19 04:33:51 WARNING  Done. 

[SSH] completed
[SSH] exit-status: 1

Troubleshooting

I think this happened because I tried running the container a couple of times by accident or perhaps stopped one unexpectedly, but after that I was unable to solve the lock.

What I've tried:

1. Checking that there is only one docker container running (the one I execute with docker run).
2. Stopping the container, removing all images, containers and volumes related to gphotos-sync.
3. Waiting 24h to re-run the command, so any short term locks would be gone.
4. Using :3.04 tag instead of :3.1.0.
5. Removing gphotos.lock, gphotos.sqlite, gphotos.sqlite-journal, gphotos.trace.
6. Restarting the server (that runs the container) and the NAS (where /storage is located).
7. Running the command with a new path: --db-path /storage/new where /storage/new is an empty folder /mnt/nas/Google/Photos/new
8. Using a few other flags instead of --progress --use-hardlinks --start-date ${start_date} - e.g. --progress only, or --index-only.
9. Leveraging the python "console" directly: went to the container python console and established a sqlite3 connection to the file path and that worked fine.

All lead to the same persistent error.

This definitely feels like a user error, but I cannot find a good fix on Stackoverflow or alike either.

[nitobuendia]

Sounds like this is probably one of the Known Issues, my bad:

Some mounted filesystems including NFS, CIFS and AFP do not support file locks and database access will fail on them.

To fix, use the parameter –db-path to specify a location for your DB on the local disk. This will perform better anyway.

I will experiment and reopen if that's not the case.

[gilesknap]

👍

[mupsje]

@nitobuendia can you please show me your whole "docker run" command you are using now? Because I have same error.

Thank you very much.

[nitobuendia]

docker-compose.yaml

services:
  gphotos:
    image:  ghcr.io/gilesknap/gphotos-sync:3.1.0
    container_name: gphotos
    restart: "no"
    stdin_open: true
    tty: true
    command: >-
        /storage
        --progress
        --db-path /db
        --albums-path /storage/albums
            --use-hardlinks
            --omit-album-date
            --album-date-by-first-photo
        --photos-path /storage/photos
        --start-date $START_DATE
    environment:
      - TZ=Asia/Singapore
    ports:
      - 'XX80:8080'
    volumes:
      - ./config:/config
      - ./db:/db
      - ./photos:/storage

Run with:

START_DATE=$(date +"%F" --date="1 month ago") docker-compose up