search

lomorage / homepage

home page source
215 stars 12 forks source link

lomorage renames files on backup #21

Open mamema opened 2 years ago

mamema commented 2 years ago

...just tested if lomorage is something for me. Unfortunately not i guess, because:

a jpg called IMG_7087.JPG will be saved as /Year/Month/Day/20220605_951.jpg

a backup solution should never do this. Even though in the lomorage IOS app, i can select the picture and view the details, i can see, that the name of the picture is IMG_7087.JPG, but on file storage side, this is not the case.

...so migrating from other solution to lomorage to prevent duplicates is a mess, same if one would migrate from lomorage to another solution.

fuji246 commented 2 years ago

Not sure why the name matters, especially when the name like IMG_7087.JPG is just an ID. The file is saved as it's except the filename, so even renaming it to different file, or import/export from/to other systems, the content of the file is untouched and all the metadata is untouched.

We use file hash to prevent duplicates, so even two files having different filename, as long as the content is the same, Lomorage will prevent that for the same user.

mamema commented 2 years ago

it matters, if, what surely is the standard, someone is coming from other solutions..... It never happens that someone can stay at one solution. Migration is part of the game. Did someone say Google Photos? So, other solutions don't touch the original files in any way. It's not the point of duplications within lomorage but instead during migration from or to another solution. For this filename matters.

coming from synology where the original names from the camera or the iphone remain eg IMG_7088.jpg going to lomorage where it is 20220605_951.jpg going to a future version where the name will be? Why is this even needed? I don't get it.

fuji246 commented 2 years ago

There has been a lot of discussion among Lomorage users on how the files should be organized on the file system, besides the naming convention, people have different opinions on the folder structure as well. We actually come up with a solution that to let the user to customize the view, https://docs.lomorage.com/docs/Usage/customized-view/ , we should be able to provide another virtual view to show the original filename, and as for the folder structure, we provide album so that the folder structure will map to folders/albums.

Will that work for you?

On Tue, Jun 21, 2022 at 12:54 PM mamema @.***> wrote:

it matters, if, what surely is the standard, someone is coming from other solutions..... It never happens that someone can stay at one solution. Migration is part of the game. Did someone say Google Photos? So, other solutions don't touch the original files in any way. It's not the point of duplications within lomorage but instead during migration from or to another solution. For this filename matters.

coming from synology where the original names from the camera or the iphone remain eg IMG_7088.jpg going to lomorage where it is 20220605_951.jpg going to a future version where the name will be? Why is this even needed? I don't get it.

— Reply to this email directly, view it on GitHub https://github.com/lomorage/homepage/issues/21#issuecomment-1162267948, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIA46YX4PG353ZVYHSCPXLVQIMXZANCNFSM5ZMUPMZA . You are receiving this because you commented.Message ID: @.***>

mamema commented 2 years ago

i don't quite understand the term "show", if the name gets modified, then it is of no help to show how it was. I have a photo library of 60.000 photos since 2001, since then. filenames are unmodified. Ten percent of those files are the "moving target" of cell phone photos which are the problem here:

fuji246 commented 2 years ago

We actually can keep the original filename in db (not supported yet, but we should be able to support that), and provide a virtual file system using WebDAV, so that we can provide a virtual view in the file system. So in short, the raw folder structure and filename is fixed, but user can choose whatever layout /view they like, to provide the flexibility for different requirements. You can copy/view those files in the file system, you should not able to tell the difference, except that we made it read-only in that view.

Which platform are you using now to try Lomorage setup? If you are using Mac/Windows, then you can easily try different virtual views here, check docs.lomorage.com/docs/Usage/customized-view.

Not quite follow the example, so you move from solution B to solution C, what do you use Lomorage for?

If you want to avoid duplicate, import everything in Lomorage, it will not save duplicate files for each user, you can try this with the web app, upload the same file twice, it will reject the second time, no matter what filename you changed to.

I do understand your concern to make sure everything is imported by checking the filename, I'm not sure what tool you use to import, we have several tools https://docs.lomorage.com/docs/Installation/lomorage-importer/. If you want to confirm it's imported, do that one more time, if it shows file already exists, it will just ignore that.

dwebfan commented 2 years ago

I guess most photo application should import photos and video and avoid duplication by one hash algorithm. Maybe I'm wrong. At least lomorage will check SHA1 before import or backup.

mamema commented 2 years ago

Not quite follow the example, so you move from solution B to solution C, what do >you use Lomorage for?

it's the backup solution, isn't it? :-) Soluton B and Solution C is the photomanagement software like piwigo or photoprism. So if i move from piwigo to photoprism i have untouched original photo share If i i've used lomorage with piwigo then lomorage starts from scratch after migration to photoprism, cause it then backed up all photos from iphone again. Of course you can then sort out whats already in photoprism. But the price is renaming the original files which i do not want. You explain it by yourself "keep original filename in the db....for a virtual view". Thats just the wrong way around. Keep original files untouched in any way and keep modified and/or hashed values in the db to prevent duplicates

dwebfan commented 2 years ago

Keep original files untouched in any way and keep modified and/or hashed values in the db to prevent duplicates

Lomorage does keep original files' actual contents untouched, and won't change contents at all. And all import operations will compare file hash, and de-duplicate if they exists.

Unfortunately, filename got changed during backup. We could add one enhancement backlog and see how to support it. BTW, besides filename, do you care folder hierarchy, for example, imported files can be stored as yyyy/mm/dd/IMG_3344.heic ?

mamema commented 2 years ago

regarding folder i have no opinion (yet). From a backup/restore point of view, a solution should be able to go back in time and provide the situation which was there back in time, so if there where different folders involved, then it should be possible to restore them. But for my usecase, i'm relying on file names

TodBonzalez commented 1 year ago

+1 to not renaming files, that surprised me