search

batrick / ceph-linode

Launch Ceph using the Linode VPS provider
GNU General Public License v3.0
13 stars 10 forks source link

Add workload for testing horizontal scaling of metadata #55

Closed sidharthanup closed 4 years ago

sidharthanup commented 4 years ago

Each client applies the workload on its own private directory. Multiple directories are created under a common directory. A new common directory is created when the no of child directories exceed 9998 to prevent the effects of directory fragmentation (for now).

Committer: Sidharth Anupkrishnan sanupkri@redhat.com

batrick commented 4 years ago

Also take a look at misc/creats.c / misc/creats.sh which just does raw open(..., O_CREATE) calls as fast as possible.

mkdir is fairly expensive in CephFS so it probably won't be as fast as creating regular files. It's still worth establishing a benchmark for mkdir though!

sidharthanup commented 4 years ago

@batrick Haven't tested this out yet but please take a look at this change. Please note I'm avoiding directory fragmentation for now (By creating a different parent just before the size threshold for dir fragmentation is reached and then continue creating child dirs under that newly created parent) , just to gauge a better sense of how metadata distribution is for distributed pin vs when run without distributed pin.