noobaa / noobaa-core

High-performance S3 application gateway to any backend - file / s3-compatible / multi-clouds / caching / replication ...
https://www.noobaa.io
Apache License 2.0
269 stars 78 forks source link

Dedup did not work fully when uploaded same file 4 times #2163

Open jeniawhite opened 7 years ago

jeniawhite commented 7 years ago

Environment info

Actual behaviour

    • Created a new bucket with policy of 1 pool with 3 nodes (all okay)
    • Uploaded 3mb file (not video/no special replicas) that created 6 parts
    • Waited that everything replicated correctly and that the blocks/chunks/parts are accessible
    • Uploaded same file ones again with a different name
    • Waited that everything replicated correctly and that the blocks/chunks/parts are accessible
    • Saw in DB that there was a full dedup (everything correct till now)
    • Uploaded same file ones again with a different name
    • Waited that everything replicated correctly and that the blocks/chunks/parts are accessible
    • Saw in DB that we created 6 more chunks which means that we did not dedup 0%
    • Afterwards did same thing with the 4th file and got a 100% dedup

Expected behavior

  1. I should see 100% all of the time and only have 6 chunks (like parts count of the first file)

Steps to reproduce

  1. Explained above

Screenshots or Logs or other output that would be helpful

(If large, please upload as attachment)

guymguym commented 6 years ago

@nimrod-becker This is too old and stale by now. We should either reproduce on master or close.

nimrod-becker commented 6 years ago

Reproduced by Liran

guymguym commented 6 years ago

I will add a unit test that writes same file repeatedly and verifies that it always finds dedup.

nimrod-becker commented 6 years ago

Per guy's comment, closing this one

guymguym commented 6 years ago

Keeping open as a reminder to commit the test

github-actions[bot] commented 4 days ago

This issue had no activity for too long - it will now be labeled stale. Update it to prevent it from getting closed.