Improved ls

[rgmiller]

Implements a new broadcast RPC that commands servers to reply with the file attribute data for all the files

Description

The new RPC is named UNIFYFS_SERVER_BCAST_RPC_METAGET. As stated above, this sends a broadcast to all servers to reply with the file attribute structs for all files they manage.

Note that this is the first (and so far, only) broadcast RPC that requires the recipients to return a significant amount of data. (Prior to this, they only data returned from a broadcast RPC was an error code if something went wrong.) This has required a fair amount of new code for gathering the results from individual children, merging them with those from the parent and propagating all of that back up the broadcast tree. See merge_metaget_all_bcast_outputs in unifyfs_group_rpc.c

Motivation and Context

This is needed to implement 'ls' like functionality. The current unifyfs-ls tool can only list files known by the server handling the ls request. With this change in place, the tool can now list files from all the server processes.

How Has This Been Tested?

Existing unit tests work (srun -N1 -n1 make check) Further testing was done with a handful of utilities to write files and then trigger the RPC.

Test environment: one Ubuntu 20.04 workstation with 3 VM's also running Ubuntu 20.04. Slurm and OpenMPI are configured on the host and VM's.

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x ] New feature (non-breaking change which adds functionality)
• [ ] Performance enhancement (non-breaking change which improves efficiency)
• [ ] Code cleanup (non-breaking change which makes code smaller or more readable)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] Testing (addition of new tests or update to current tests)
• [ ] Documentation (a change to man pages or other documentation)

Checklist:

• [x ] My code follows the UnifyFS code style requirements.
• [ ] I have updated the documentation accordingly.
• [x ] I have read the CONTRIBUTING document.
• [ ] I have added tests to cover my changes.
• [x ] All new and existing tests passed.
• [x ] All commit messages are properly formatted.

[rgmiller]

As is typical of my pull requests, there are a handful of comments marked as "TODO" that describe questions I have about the proper way to accomplish something. In particular, I'd like some input regarding the comments on lines 182 and 262 of unifyfs_group_rpc.c.