search

bpatrik / pigallery2

A fast directory-first photo gallery website, with rich UI, optimized for running on low resource servers (especially on raspberry pi)
http://bpatrik.github.io/pigallery2/
MIT License
1.81k stars 210 forks source link

Support XMP sidecar files #156

Open properlypurple opened 4 years ago

properlypurple commented 4 years ago

Is your feature request related to a problem? Please describe. I use Digikam and Darktable both to manage my photos, and I like keeping the metadata in XMP sidecar files, which makes it a lot easier to maintain interoperability, as well as file syncing and versioning for backup. Currently Pigallery2 reads tags directly from the image file, but ignores XMP sidecar files, thus making it not very useful for tags/faces, etc

Describe the solution you'd like When the metadata is loaded, look for and load from xmp sidecar files as well as the image.

Pigallery2 is the best tool I have found yet, to maintain a read-only gallery for my already managed filesystem hierarchy of photos. I already have a syncing and backup system in place, as well as the apps to manage the photos. Support for sidecar files will make this feature complete for me. I'm happy to provide sample images and files for development/testing.

bpatrik commented 4 years ago

This is actually a great idea.

Would you be interested contributing this to the gallery? I can give you some pointers, shouldn't be that complicated. If not, I'll live this issue open and once I catch up with my life, I'll implement it.

properlypurple commented 4 years ago

Thanks @bpatrik . My javascript is not great, but I'd love to give this a try. I'll set up a dev environment some time this week and will look into this. I'd love any pointers you can give me.

bpatrik commented 4 years ago

Hi, Sorry for the late reply. If you still have the time to do this feature here are the pointers:

dev env: 1) download source 2) npm install to get all dependencies 3) npm run run-dev (you can kill the process once it is finished building the fronted as you not going to change that) 4) npm run build-backend this only compiles the typescript files, you can also run tsc -w if zou have typescript installed globally 4) npm start to run the server

Where to change: The metadata loading is happening in the MetadataLoader.ts file: https://github.com/bpatrik/pigallery2/blob/af012060ec420fabd8938797a38b6210dc40d3a4/src/backend/model/threading/MetadataLoader.ts#L80 I'm reading the beginning of the photo file here: https://github.com/bpatrik/pigallery2/blob/af012060ec420fabd8938797a38b6210dc40d3a4/src/backend/model/threading/MetadataLoader.ts#L84-L85 Then I'm using multiple libraries to parse the metadata (unfortunately there is not a good what that can do everything on its own). I'm using these libs: ts-exif-parser, ts-node-iptc, exifreader, image-size. Please check if any of those support reading the sidecar files, before adding a new dependency to the project.

If you update the metadata variable in the loadPhotoMetadata(fullPath: string) function with the data from the sidecar files, than the site will handle it properly, you do not need to do any other changes.

properlypurple commented 4 years ago

I did some tinkering, and it looks like none of the existing dependencies support reading xmp files. I was able to find this https://github.com/PhilipF5/xmp-sidecar

but I wasn't able to get it working properly. I'm going to start taking out time to try and work for this, but I have no high hopes. Thank you for the detailed instructions(maybe they should go in the readme, or a wiki page.)

mcdamo commented 3 years ago

I'm interested in supporting XMP sidecars because this will allow metadata for videos such as tags and descriptions.

xmp-sidecar doesn't appear to decode much of the metadata so you would have to manually comb through the raw xml structures anyway.

exiftool is probably a better option, it reads and decodes all file types including XMP sidecars and is interfaced by packages such as node-exiftool and exiftool-vendored.

bpatrik commented 3 years ago

I'm not super happy, but I think Its okay if we introduce a new dependency for sidecars. My main issue in #277 that a single photo is getting read 4 times to get all the data. Sidecars live in a separate files that needs to be opened at least once independently from the used lib.

On the other hand, @mattiasw already offered his help in https://github.com/bpatrik/pigallery2/issues/277#issuecomment-829881480, maybe he is also willing to support sidecars with https://github.com/mattiasw/ExifReader.

exiftool looks promising. I haven't looked into it in depth, from two libs that you mention, exiftool-vendored looks better (more stars, actively developed, more contributors, etc.). Using native code is awesome, but the big challenge with that they sometimes do not compile to arm. (I also have issues with the new sharp, I cannot update to the latest as that does not compile to arm)

I'm very welcome a PR if you have the time @mcdamo.

mattiasw commented 3 years ago

Do you have an example XMP sidecar file that I can look at for ExifReader? I'm wondering how complicated it would be to add that functionality.

mcdamo commented 3 years ago

Do you have an example XMP sidecar file that I can look at for ExifReader? I'm wondering how complicated it would be to add that functionality.

XMP is a standard, you can generate from exiftool or find samples all over the web.

bpatrik commented 3 years ago

@mattiasw there is also a similar issue at exifr github with some example: https://github.com/MikeKovarik/exifr/issues/62

kagahd commented 1 year ago

Since exifr#62 is able to read sidecar files, will reading XMP sidecar files be supported by PiGallery2 in the near future? That would be awesome!

bpatrik commented 1 year ago

Its on my (unwritten) roadmap, but I cant give a deadline.

Ultra2D commented 1 year ago

Just started looking into hosting some of my photos online. So far I am really digging Pigallery2. My workflow would be Digicam for face detection and geolocation, then Darktable for further editing. I'd like to keep the original files as is, and use sidecars.

If you update the metadata variable in the loadPhotoMetadata(fullPath: string) function with the data from the sidecar files, than the site will handle it properly, you do not need to do any other changes.

Are you saying that if all metadata is contained in the sidecar XMP, I could 'just' try to load from the sidecar instead of the photo? Hacking something like that in as a test is about my level of 'coding' .

p2kmgcl commented 4 months ago

Anyone working on this? I'd be happy to contribute to this feature

Wihlelm commented 4 months ago

I was also expecting an update on this ticket for many months, and a few days ago, I tried the docker image pigallery2:edge-alpine. As far as I can see, the sidecar files are now perfectly supported...

grasdk commented 3 weeks ago

Yeah. Should be supported. Just discovered this ticket now. @bpatrik I guess this ticket can be closed.