minio / dsync

A distributed sync package.
Apache License 2.0
400 stars 41 forks source link

Change dsync to support more client objects per instance #50

Closed harshavardhana closed 6 years ago

harshavardhana commented 6 years ago

This is precursor change to support 16 > slots in erasure coded set up under minio. The new style of slots handling allows minio to expand the storage from 16 disks to multiple of 16 disks allowing the namespace to scale to beyond 16 disks.

Locking on the other hand in current style expects one dsync instance per node, this assumption should be changed to accomodate the new change of single server exporting more than 16 disks.

Each slot will have its own dsync allowing objects to be independently locked and accessed per slot. Since object is hashed deterministically to a slot, each slot holds its own locking.

codecov-io commented 6 years ago

Codecov Report

Merging #50 into master will not change coverage. The diff coverage is 91.22%.

Impacted file tree graph

@@          Coverage Diff           @@
##           master     #50   +/-   ##
======================================
  Coverage    86.3%   86.3%           
======================================
  Files           3       3           
  Lines         241     241           
======================================
  Hits          208     208           
  Misses         27      27           
  Partials        6       6
Impacted Files Coverage Δ
dsync.go 100% <100%> (ø) :arrow_up:
drwmutex.go 85.95% <88.63%> (+0.07%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a26b9de...e4185d8. Read the comment docs.

harshavardhana commented 6 years ago

We need to make this change nevertheless. Whoever uses would have vendorized the package so not a problem.

harshavardhana commented 6 years ago

Of course this implicates an API change so we need to mention that in the README once we merge it.

What would like to be mentioned in the README.md ?

fwessels commented 6 years ago

Something to the effect of that the Init function has been deprecated in favor of a New method and that the creation of an DRWMutex now takes a *Dsync parameter.

harshavardhana commented 6 years ago

Something to the effect of that the Init function has been deprecated in favor of a New method and that the creation of an DRWMutex now takes a *Dsync parameter.

Yeah i don't see it in current README.md - will just add a new blockquote

harshavardhana commented 6 years ago

Added README.md mention please review again @fwessels @krisis