bpatrik
commented
2 years ago My guess is that mounting hides the folder modification date, so the app thinks it never changed.
Try this setting this to a higher value (so the app will more aggressively reindex):
When mounting
/app/data/imagesusing the rclone docker volume plugin, new files added to the directory (after pigallery starts) are never scanned/displayed until the database is reset. Simply indexing does not help.Notably, new files can still be found in pigallery by direct paths (i.e. going to
pigallery.example.com/gallery/newdirprompts a rescan and displays all images correctly, butnewdiris never shown atpigallery.example.com/galleryuntil the database is reset).The problem appears to be in the combination of pigallery and rclone: mounting local directories works fine, so sorry if this question/report is misplaced. However, in defense of rclone itself, this exact volume on the same machine works perfectly well with other services, such as filebrowser and Nextcloud even, pigallery alone seems to have issues.
I'm wondering: what could be causing this issue in pigallery that is not present in other services? My understanding is that metadata on the mounted volume behaves the same, regardless of what docker volume driver is used.
Edit: for context, briefly, the rclone docker plugin allows to seamlessly mount a great number of cloud storage providers/protocols (from FTP, to Dropbox, to S3) as volumes into containers.
Environment:
bpatrik/pigallery2:latest(00328c4b2f92)docker-compose.yml
``` yaml volumes: rclone-volume: driver: rclone driver_opts: remote: 'myremote:' allow_other: 'true' vfs_cache_mode: 'full' services: pigallery2: image: bpatrik/pigallery2:latest environment: - NODE_ENV=production volumes: - ./pigallery2/config:/app/data/config - ./pigallery2/db:/app/data/db - ./pigallery2/tmp:/app/data/tmp - rclone-volume:/app/data/images expose: - 80 restart: always caddy: ... ```Full config
All defaults with the exception of log level "silly". ``` json { "Server": { "sessionSecret": [ "42dbb20a1da200a70a3b6944c62a5cf74981f836eb6e5be528b575fb1383788761588d15dd00d69d37cf0eaf029765aaa589d1827b48ca56124e74151c1512178d104e0ca07bc83edd73739751624bb1dc9112f2a8386fd2882ff448920736813d45e16e3cb0c917fca66741b90f442369d18d39d817e3898fbe82be7af8a33935aa57fc07caedcb081a090e871d1aee4bc1a3869cd3d9f4dfffa1c210c6f29cbccb08e5d78d3321688f30da1280757002cfb1a3f11beaad6c523345a97c054a20a7706e75de3f1ef3fd17aaf97b89503d2075d16c39c91e7e03390b2df3dda74ebb7c34b9ca2f167d356de5b35bdafb0bc66637a1f1583ff3a5c8d293d52eb4", "e7c26eb2f19605d70e1de39401c75637ebd7210bd2f9c845bf4ead3b2ad0cd98f85377ac2efe97ccecaaac3a031bd3559cc8e86d1bf1be4beecca35bc9ab8caf07da3a0738b09fd517cf1bcdfaaeee4a1864f34ce80e7700a4fc678fe05eed14bc05338259bab18dd196e47a06835f4a22f8a8bcc023576b3d94babe87c743f15efa2776736a4e6e628e5c9fc3b2f70daaeec5d17eeee452985f33f089f8c7e7ab642b096bf26b5e912dae7e420cd5da72ca0e7d4ac3118625b772d19cd097ea52a4d9d06cf94cabdb0f32d0609cef8be096243eb3c77439a23a45e558c324c76de4fade9a36946afd8133899e2005d46127cca64db1d8a3e9927319b11be1d3", "e1ea47e4ab494e6adb2573a736f0346b7206b65bae74e8d800e8dd768c374e9cc9434597cbf4faaf73f5610307c83ac9b7fee9cb6cb77a5863b37a4ab947da6df0d5bdb59f161d3502c0c77b42af22ebf17c35daa2a9f5a9c849bb49289b5df6a80cbc3ea2975af2c5c6ab1f9b1995a3f72f9f854d0a71ad8c3035ceb105ce806447e01f7143028cd87e0a0a6bbaf5cbf9c8a567609c8da5e49cca4236202be571547cbae8226849a4e14006aa894e9d6a5d499eafe3d29c3fbace5af8a6f3c8dd910011c391939857b33605ac20d781cbb3ab8ec3962158bb1670f79fb45f76e3425d358424015aa240fcd722ad69e0ee17e53b3b96bb31e248456df7359934" ], "port": 80, "host": "0.0.0.0", "Media": { "//[folder]": "Images are loaded from this folder (read permission required)", "folder": "/app/data/images", "//[tempFolder]": "Thumbnails, converted photos, videos will be stored here (write permission required)", "tempFolder": "/app/data/tmp", "Video": { "transcoding": { "bitRate": 5242880, "resolution": 720, "fps": 25, "codec": "libx264", "format": "mp4", "//[crf]": "Constant Rate Factor. The range of the CRF scale is 0–51, where 0 is lossless, 23 is the default, and 51 is worst quality possible.", "crf": 23, "//[preset]": "A preset is a collection of options that will provide a certain encoding speed to compression ratio", "preset": "medium", "//[customOptions]": "It will be sent to ffmpeg as it is, as custom options.", "customOptions": [] } }, "Photo": { "Converting": { "//[onTheFly]": "Converts photos on the fly, when they are requested.", "onTheFly": true, "resolution": 1080 } }, "Thumbnail": { "//[qualityPriority]": "if true, photos will have better quality.", "qualityPriority": true, "personFaceMargin": 0.6 } }, "Preview": { "SearchQuery": { "type": 100, "text": "" }, "Sorting": [ 6, 4 ] }, "Threading": { "//[enabled]": "App can run on multiple thread", "enabled": true, "//[thumbnailThreads]": "Number of threads that are used to generate thumbnails. If 0, number of 'CPU cores -1' threads will be used.", "thumbnailThreads": 0 }, "Database": { "type": "sqlite", "dbFolder": "/app/data/db", "sqlite": { "DBFileName": "sqlite.db" }, "mysql": { "host": "localhost", "port": 3306, "database": "pigallery2", "username": "", "password": "" }, "//[enforcedUsers]": "Creates these users in the DB if they do not exist. If a user with this name exist, it wont be overwritten, even if the role is different.", "enforcedUsers": [] }, "Sharing": { "updateTimeout": 300000 }, "//[sessionTimeout]": "unit: ms", "sessionTimeout": 604800000, "Indexing": { "cachedFolderTimeout": 3600000, "reIndexingSensitivity": "low", "//[excludeFolderList]": "If an entry starts with '/' it is treated as an absolute path. If it doesn't start with '/' but contains a '/', the path is relative to the image directory. If it doesn't contain a '/', any folder with this name will be excluded.", "excludeFolderList": [ ".Trash-1000", ".dtrash", "$RECYCLE.BIN" ], "//[excludeFileList]": "Any folder that contains a file with this name will be excluded from indexing.", "excludeFileList": [] }, "//[photoMetadataSize]": "only this many bites will be loaded when scanning photo for metadata", "photoMetadataSize": 524288, "Duplicates": { "listingLimit": 1000 }, "Log": { "level": "silly", "sqlLevel": "error", "logServerTiming": false }, "Jobs": { "//[maxSavedProgress]": "Job history size", "maxSavedProgress": 10, "scheduled": [ { "name": "Indexing", "jobName": "Indexing", "config": { "indexChangesOnly": true }, "allowParallelRun": false, "trigger": { "type": "never" } }, { "name": "Preview Filling", "jobName": "Preview Filling", "config": {}, "allowParallelRun": false, "trigger": { "type": "never" } }, { "name": "Thumbnail Generation", "jobName": "Thumbnail Generation", "config": { "sizes": [ 240 ], "indexedOnly": true }, "allowParallelRun": false, "trigger": { "type": "after", "afterScheduleName": "Preview Filling" } }, { "name": "Photo Converting", "jobName": "Photo Converting", "config": { "indexedOnly": true }, "allowParallelRun": false, "trigger": { "type": "after", "afterScheduleName": "Thumbnail Generation" } }, { "name": "Video Converting", "jobName": "Video Converting", "config": { "indexedOnly": true }, "allowParallelRun": false, "trigger": { "type": "after", "afterScheduleName": "Photo Converting" } }, { "name": "Temp Folder Cleaning", "jobName": "Temp Folder Cleaning", "config": { "indexedOnly": true }, "allowParallelRun": false, "trigger": { "type": "after", "afterScheduleName": "Video Converting" } } ] } }, "Client": { "applicationTitle": "PiGallery 2", "publicUrl": "", "urlBase": "", "Search": { "enabled": true, "searchCacheTimeout": 3600000, "AutoComplete": { "enabled": true, "targetItemsPerCategory": 5, "maxItems": 30, "cacheTimeout": 3600000 }, "maxMediaResult": 10000, "//[listDirectories]": "Search returns also with directories, not just media", "listDirectories": false, "//[listMetafiles]": "Search also returns with metafiles from directories that contain a media file of the matched search result", "listMetafiles": true, "maxDirectoryResult": 200 }, "Sharing": { "enabled": true, "passwordProtected": true }, "Album": { "enabled": true }, "Map": { "enabled": true, "//[maxPreviewMarkers]": "Maximum number of markers to be shown on the map preview on the gallery page.", "maxPreviewMarkers": 50, "useImageMarkers": true, "mapProvider": "OpenStreetMap", "mapboxAccessToken": "", "customLayers": [ { "name": "street", "url": "" } ] }, "RandomPhoto": { "//[enabled]": "Enables random link generation.", "enabled": true }, "Other": { "customHTMLHead": "", "enableCache": true, "enableOnScrollRendering": true, "defaultPhotoSortingMethod": "ascDate", "//[enableDirectorySortingByDate]": "If enabled directories will be sorted by date, like photos, otherwise by name. Directory date is the last modification time of that directory not the creation date of the oldest photo", "enableDirectorySortingByDate": false, "enableOnScrollThumbnailPrioritising": true, "NavBar": { "showItemCount": true }, "captionFirstNaming": false, "enableDownloadZip": false, "//[enableDirectoryFlattening]": "Adds a button to flattens the file structure, by listing the content of all subdirectories.", "enableDirectoryFlattening": false }, "authenticationRequired": true, "unAuthenticatedUserRole": "Admin", "Media": { "Thumbnail": { "iconSize": 45, "personThumbnailSize": 200, "thumbnailSizes": [ 240, 480 ] }, "Video": { "enabled": true }, "Photo": { "Converting": { "enabled": true }, "//[loadFullImageOnZoom]": "Enables loading the full resolution image on zoom in the ligthbox (preview).", "loadFullImageOnZoom": true } }, "MetaFile": { "//[gpx]": "Reads *.gpx files and renders them on the map.", "gpx": true, "//[markdown]": "Reads *.md files in a directory and shows the next to the map.", "markdown": true, "//[pg2conf]": "Reads *.pg2conf files (You can use it for custom sorting and save search (albums)).", "pg2conf": true }, "Faces": { "enabled": true, "keywordsToPersons": true, "writeAccessMinRole": "Admin", "readAccessMinRole": "User" } } } ```Full log
1. Start new server with one directory (`/labrador/`) 2. Log in 3. Push a new directory alongside the existing one (`/dingo/`) 4. Renew page 5. _Run now: Indexing_ in settings 6. _Run now: Database Reset_ in settings 7. Go back to `/gallery`, refresh page ``` log 7/2/2022, 6:53:25 PM[INFO_][server] running diagnostics... 7/2/2022, 6:53:25 PM[INFO_][SQLConnection] Updating database scheme 7/2/2022, 6:53:26 PM[VERBS][server] using config from /app/data/config/config.json: