LLNL / UnifyFS

UnifyFS: A file system for burst buffers
Other
105 stars 31 forks source link

Changes to support Node Local extents. #725

Closed hariharan-devarajan closed 1 year ago

hariharan-devarajan commented 2 years ago

Description

Currently, unifyfs clients read node-local data from the server thread which is a bottleneck. In the case of Laminated files, we can fetch the node-local extents from the server and perform read locally. With the new proposed changes

In these cases, we do only one RPC per request group and fetch extents for all laminated files.

Motivation and Context

This feature would enable faster reads by clients in the case of laminated files. Some examples of this are,

How Has This Been Tested?

In the API test case, I have enabled node_local_extents to true. This works with existing tests well as for non-laminated files this optimization is not applied. Whereas for laminated files it is.

Types of changes

Checklist:

hariharan-devarajan commented 2 years ago

@hariharan-devarajan Thanks, this is getting really close to being ready - just a few more requested changes. After that, we'll need to rebase/squash down to a single commit.

I have rebased it :)