skoobasteeve / rayagainstthemachine.net

A personal blog built with Jekyll and the minimal-mistakes theme.
https://rayagainstthemachine.net
0 stars 0 forks source link

linux%20administration/nextcloud-photos/ #4

Open utterances-bot opened 3 years ago

utterances-bot commented 3 years ago

Better Nextcloud Photo Albums with Preview Generator and Exiftool - Ray Against the Machine

If you host and use a Nextcloud server, you know that it’s good at many things. Unfortunately, displaying photos is not one of them.

https://rayagainstthemachine.net/linux%20administration/nextcloud-photos/

jbenitosantoni commented 3 years ago

Hello, this a very useful tutorial.

But I had a problem, the exiftool command used here didnt work for me when I need to change the date of .mov or .mp4 files.

In my case I had to use this command instead "-FileModifyDate<CreateDate" as they dont have the DateTimeOriginal original tag.

skoobasteeve commented 3 years ago

@jbenitosantoni Thanks for the feedback! I'll look at adding some simple if/then logic to the script for video files which don't have the same EXIF data as photos.

fly800 commented 3 years ago

Hi, this is great.

I use Nextcloud as a jail in TrueNAS. I had to adapt it to my setup.

I have a couple problems: To start of first. In the Photo Sorting portion of your tutorial using the script for servers. It writes all photo filenames with metadata "create date" but in nextcloud the filenames remain with the old filenames.

root@nextcloud-jail:~ # ls -al /usr/local/www/nextcloud/data/steven/files/Photos/'Steven Ipad'/2019/04/ total 5229 drwxr-xr-x 2 www www 6 Mar 14 13:07 . drwxr-xr-x 6 www www 6 Mar 9 18:18 .. -rw-r--r-- 1 www www 1307951 Apr 27 2019 2019-04-27_23.21.17-1.jpg -rw-r--r-- 1 www www 1307951 Apr 27 2019 2019-04-27_23.21.17.jpg -rw-r--r-- 1 www www 1297818 Apr 27 2019 2019-04-27_23.21.25-1.jpg -rw-r--r-- 1 www www 1297818 Apr 27 2019 2019-04-27_23.21.25.jpg

Files in Nextcloud webpage 20190427-2321-0145.jpg 19-04-27 23-21-17 0135.jpg 19-04-27 23-21-24 0145.jpg 20190427-2321-0135.jpg

Any ideas why this is happening. I think maybe mysql db is not updating with the new filename.

Thanks for help.

skoobasteeve commented 3 years ago

@fly800 Make sure the last line of the server script is running successfully on your machine.

sudo -u www-data php "$nextclouddir"/occ files:scan

This command is what updates the Nextcloud db to recognize the name changes. Try running it on its own and see if you get a result.

fly800 commented 3 years ago

Thanks Ray,

I had to use <--all> at the end to make it work.

In my case sudo -u www php "$nextclouddir"/occ files:scan --all

Great works now thanks. Steven Vézina

On Sun, Mar 14, 2021 at 4:05 PM Ray Lyon @.***> wrote:

@fly800 https://github.com/fly800 Make sure the last line of the server script is running successfully on your machine.

sudo -u www-data php "$nextclouddir"/occ files:scan

This command is what updates the Nextcloud db to recognize the name changes. Try running it on its own and see if you get a result.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/skoobasteeve/skoobasteeve.github.io.2/issues/4#issuecomment-798961201, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATHIJMSGVRXH4VUTRFBTZ43TDUCHLANCNFSM4WVQICGA .

ColinMacLaren commented 3 years ago

Thanks for your guide. I am using gphtosync to pull my pictures of Google Photos. So a couple fo tips for those that are doing the same thing:

All pictures synced from GPhotos are JPEGs, but they keep their original file extension. This is going to mess things up in both exiftool and media players. So I change all extensions to .jpg

find "$albumdir" -type f -name '.PNG' -print0 | xargs -0 rename 's/.PNG$/.jpg/' find "$albumdir" -type f -name '.png' -print0 | xargs -0 rename 's/.png$/.jpg/' find "$albumdir" -type f -name '.HEIC' -print0 | xargs -0 rename 's/.HEIC$/.jpg/' find "$albumdir" -type f -name '.heic' -print0 | xargs -0 rename 's/.heic$/.jpg/'

@jbenitosantoni Vidoes files and even some pictures have no DateTimeOriginal set, so I do a

exiftool -if 'not defined $datetimeoriginal' "-FileModifyDate>DateTimeOriginal" -r "$albumdir" -overwrite_original

first. No I can continue with the script from Ray.

exiftool "-filemodifydate<datetimeoriginal" -r "$albumdir" -overwriteoriginal exiftool '-FileName<DateTimeOriginal' -r -d "%Y-%m-%d%H.%M.%S%%-c.%%e" "$albumdir"

jcm4atx commented 3 years ago

Ray,

This is great. It takes exiftool a step further and uses tools I didn't realized Nextcloud even had.

I like sorting my photos into directories based on date. I show how to that here.

samgarmany commented 2 years ago

I keep having it say Warning: No writable tags set from /media/myCloudDrive/ncdata/data/ncp/files/Photos/ every time I run the script what could be going wrong, I may have set up the script incorrectly

bpottle1 commented 2 years ago

@Ap0them I am having the same issue. Have you found a solution yet?

samgarmany commented 2 years ago

@Ap0them I am having the same issue. Have you found a solution yet?

Yea I just gave up on Nextcloud & went to samba & photosync

skoobasteeve commented 2 years ago

@Ap0them @jcm4atx Typically this error appears when exiftool finds a file that doesn't contain image metadata, like a video or document. The message will show for each applicable file but the script should still work on your other files with no output. Can you confirm whether or not your image files are being updated despite the error?

bpottle1 commented 2 years ago

@skoobasteeve I think it is actually mostly working for me. It may have just been taking longer to update filenames in the browser for some reason but they are looking good now. Something to note though, it looks like any photos I had that were in HEIC format (for a live photo from iphone) have now been separated into jpg and mov files. The still image jpeg looks like it updated filenames properly, but the mov file of the live photo doesn't have any metadata associated with it. Do you know if it is possible to insert metadata into a file based on the current filename? Most of my photos have a date and time already in the filename, so they just need metadata inserted based on that.

Thanks for the awesome scripts by the way!

skoobasteeve commented 2 years ago

@bpottle1 Great to hear! The issue is that video files don't have any EXIF metadata associated with them, they're just treated as standard files with the a modified date. You can manually change the modified date of a file using the touch command so there's probably a creative way to script it for live photos. I'll take a look when I have more time!

vspaziani commented 2 years ago

@skoobasteeve @bpottle1 I modified the code so that it didn't change the file names when it was scanning. The Live Photo preview on Nextcloud appears to look if there are two files that have the same name one with a MOV extension and combines them into a Live Photo preview. Alternatively, I was thinking about setting it up so that whatever the JPG gets changed to try to change the MOV file to the same... but I haven't gotten a chance to test that one out.

bpottle1 commented 2 years ago

Yeah I was actually working on the same problem haha. I figured out how to make it not rename it if it is a Live Photo, but my OCD demands that they all have the same naming scheme 😂 so I’ll keep looking at it as well.

RobMahn commented 2 years ago

I was already headed down the exiftool path to properly date sort my images. Unfortunately, I scanned old family pictures and gave them appropriate historical exif dates, but you cannot set a Windows file modification date prior to 1980. I see some pokes at getting Nextcloud to use the exif created date directly, but it appears not to be there yet.

skoobasteeve commented 2 years ago

@RobMahn I feel your pain! Nextcloud Photos would be such a killer app if it supported EXIF data. For your case, seems like renaming the files based on the EXIF data and sorting by filename is your best bet.

dmalanij commented 2 years ago

Thanks a lot for this guide, very useful for a NextCloud noob and enthusiast as me.

A minor comment, according to NextCloud official documentation for previews, the jpeg_quality should be set vía occ (just reviewed and it says the same since version 17 to current 24).

pjlinden commented 1 year ago

I've had to modify it to make it work on docker containers with Ansible but this was very helpful. Thanks!

belutz88 commented 1 year ago

Hello, I want to try the exiftool script on my server following your tutorial and before doing so I have a question. I have all my photos organized by subfolders in a main folder, my question is if I put the script in the root folder the program is able to recognize the photos in the subfolders or do I have to put the script in all the folders with photos. Thank you!

vspaziani commented 1 year ago

@belutz88 the script will go through all of the subfolders, when you run the script you specify the root directory to start in i.e. ./photo-cleanup.sh "/home/raylyon/Nextcloud/Photos"

belutz88 commented 1 year ago

thanks for the reply vspaziani After having everything and executing the script, this appears: Warning: No write labels have been set since /..... Why is this happening?

belutz88 commented 1 year ago

But my photos do have metadata. I have tried with this command on a photo. exiftool -time:all -s 'photo name' response: FileModifyDate : 2012:12:03 20:28:45+01:00 FileAccessDate : 2022:10:30 12:28:43+01:00 FileInodeChangeDate : 2022:10:29 15:10:02+02:00 ModifyDate : 2007:12:15 16:55:11 DateTimeOriginal : 2007:12:15 16:55:11 CreateDate : 2007:12:15 16:55:11

but it keeps telling me: Warning: No writable tags set from /

velascog commented 1 year ago

Great tutorial.

I have the images in external drives. Where does preview generator store the previews, along w/ the original images?

keesfluitman commented 1 year ago

I have nextcloud in a docker container on unraid. It says exiftool not found. What is best to do? The app memories (of nextcloud) does have exiftool, but if I start the script from within that folder, it still says it can't find exiftool. So it needs to be installed within my container somehow.

dmalanij commented 1 year ago

@keesfluitman I'm also working with nextcloud container image. I haven't yet got to the point of exif usage but in any case the approach should be suitable for you: basically you can create your own image based on the official one adding to it whatever you need there. For instance:

FROM nextcloud:latest

RUN apt-get update \
    && apt-get install -y libimage-exiftool-perl

Then building the image locally with something like:

docker build -t nextcloud-with-tools:latest .

And start the container with this new custom image. It will depend on how is your setup how can you automate some of this for keeping it updated.

I'm assuming you have the basic grasps for docker usage, hope this helps. Cheers

keesfluitman commented 1 year ago

thanks. But I run Unraid and found that using a plugin (NERD) i could easily install exiftool directly on unraid. Then running it (or even setting a script to run regularly) is possible. I just had all my video files that were in the same folder changed wrongly though, but I guess I just need to figure out how to change that properly...

dmalanij commented 1 year ago

Ohh, nice! I'm kind of fixed with the idea of not using the host for anything and leave Nextcloud related stuff within its container, but I know is my own poison :)

Pity for the error when applying the change but probably you can restore that soon. Cheers

KonTiki1957 commented 1 year ago

Hi, I liked your work and altough I'm not very expert I decided to give it a try but I'm failing with your script because of permission issues, for each photo I get: Warning: No writable tags set from /home/nextcloud-folder/data/ ... what am I doing wrong? I even tried with "sudo -u www-data exiftool ..." but it doesn't help

skoobasteeve commented 1 year ago

@KonTiki1957 exiftool throws that error whenever it encounters a non-image file. It means that the file it tried to update the metadata for doesn't have the relevant metadata tag available. In theory, the script should have still worked on all the image files it found during the process.

If you want to be sure, point the script at a specific directory containing your photos, not the entire /data directory.