photoprism / photoprism

AI-Powered Photos App for the Decentralized Web 🌈💎✨
https://www.photoprism.app
Other
34.62k stars 1.9k forks source link

Export: Download all (death of admin) #3440

Open DunklerPhoenix opened 1 year ago

DunklerPhoenix commented 1 year ago

Describe what problem this solves and why this would be valuable to many users

I host photoprism instances for some users. So what would happen if I die or anything happens to me. Many users are not expierienced enough to setup the sync feature.

Describe the solution you'd like

It would be great if there is an export page to trigger a total export. The user clicks a button and all images get zipped in background and the user can come back to download the zip file(s).

If not enough space is left, photoprism will only zip enough files that it fits on the disk and after the user came back he can download the first stack of files and choose to continue afterwards. Then the old zips get deleted from the server and photoprism generate the next stack of zip files.

EDIT: I also think that DSGVO requires a feature gor the user to totally export all data, doesnt it?

lastzero commented 1 year ago

If you really want to download everything, we can just stream the files as zip. However, be prepared that downloading all the files this way can take a very long time, so the chance of errors is quite high unless you do it locally or with an extremely stable internet connection that never fails. You will also need extra disk space and CPU to decompress the zip file. For these reasons, I would strongly recommend that you sync the files via WebDAV instead, which is natively supported by all modern operating systems. If a file fails, you can selectively re-download it instead of having to start from scratch.

Implementing a web download for index database dumps makes sense though, if properly secured to prevent unauthorized access.

DunklerPhoenix commented 1 year ago

Yeah I know. But the main problem is that the sync is too complicated for unexperienced users.

And the the thing with the zip files... Thats the reason why I wrote the thing with the export page that prepare anything with multiple zip files. (Like google takeout)

lastzero commented 1 year ago

There is no reason to waste disk space with zip exports. We currently only do this when dealing with a selection of files, as the GET query may not be large enough to reference all IDs. Complete albums can be streamed, for example. We could also stream based on a range of file IDs, so that each ZIP file contains, for instance, 1000 files. If you download via WebDAV, the transferred files are also compressed, but transparently, so you don't have to manually decompress them after downloading. Doing this and then merging all the ZIP files into a single folder again (without messing up the nesting if your mouse isn't very precise) is quite tedious if you think about it.

Users who self-host PhotoPrism will typically have direct access to the file system by design - either on the local computer or via NFS/CIFS if it is a NAS.

lastzero commented 1 year ago

Here's how Google "offers" you to download your pictures. I would say this is a step backwards and not very practical:

20230604_065037

DunklerPhoenix commented 1 year ago

You could choose the Container size there. It goes up to 50GB / zip (or tar.gz)

lastzero commented 1 year ago

The person who posted this then failed to do that (it's the maintainer of the Caddy Web server). Also, some filesystems like FAT have a 2 GB per file limit, which I guess is the reason that this is the default. Otherwise, it could result in download errors, which ultimately leads to support requests when Windows users are not aware of this limitation.

DunklerPhoenix commented 1 year ago

Again: this Feature Request is mainly for a worst case where an absolute noob should be able to save his own data. So imagine a really stupid person and say he should transfer his data via webdav. They dont even know what this is

lastzero commented 1 year ago

We could offer a Windows installer for WebDAV and call it Photo Drive. That's what Deutsche Telekom and others do. They all use it, just name it differently. Wouldn't be surprised if the Google Drive OS "plugin" also uses it under the hood.