search

sublinks / sublinks-api

MIT License
68 stars 17 forks source link

Name the uploaded images different #387

Open Pdzly opened 1 month ago

Pdzly commented 1 month ago

Currently if a user uploads "A cute teddybear.jpg" it stays "A cute teddybear..." on the server this could lead to duplicates / errors if another "A cute teddybear" gets uploaded.

I would recommend to name them "{person.id}-{file.name}-{date}" So it could be identified just by looking into the file name who uploaded it and when.

jgrim commented 1 month ago

I don't have any issues uploading files with the same name. Have you noticed this issue somehow?

Pdzly commented 1 month ago

@jgrim its rather not good to have the images named like they do originally.

Best it should be directly identifiable e.g. {uploader.id}-{random-hash}.png/jpg/webm

  1. To be able to identify from whom it came, 2. Dont have the mess with duplicates names. Idk what pictrs do with them, override them?

Edit: If lemmy has something different, pictrs just ignores the names and deduplicates it with image file hashes

We dont know if they have it in any meta data but idk

Security-Chief-Odo commented 2 weeks ago

Pictrs doesn't store the actual image file itself, and nowhere on the filesystem or sled repo, is the uploaded file name. The identifier EG: e47-35c3e-95038ded-7a46-4d1a is in the pictrs Database to the filename/path in the Lemmy Database. pictrs uses sled for it's database. Lemmy code connects with it and translates as LEMMY needs, with it's requests/Database.

In pictrs there is no such thing as a 'duplicate filename'. There is a concept of 'duplicate file or file pieces'.

Sled uses modern b-tree techniques such as prefix encoding and suffix truncation for reducing the storage costs of long keys with shared prefixes. If keys are the same length and sequential then the system can avoid storing 99%+ of the key data in most cases, essentially acting like a learned index

ls equiv on the object storage for filenames:

    73689 Images/002/449/684/2bd0ad8e-5c54-4447-891b-e9eaf35c721b
  2348777 Images/002/449/684/6a5955c1-0171-4b43-b6fb-b3b0d0e6673b
    15996 Images/002/449/685/cd0e98a5-892a-41ac-a271-f038c7fcb2bf
     2922 Images/002/449/686/2cc9a21f-e982-4aed-bf4e-6a3963048564