Creates a "core" library that implements all the shared functionality across APIs

[ilumsden]

Previously, each API re-implements the main (or "core") functionality of DYAD, such as Flux KVS put/get operations and Flux RPC calls. This PR abstracts all this shared functionality into an underlying libdyad_core.so library with the following API:

// Store all the data needed to use DYAD
struct dyad_ctx;
typedef struct dyad_ctx dyad_ctx_t;

// Store information fetched from the Flux KVS
struct dyad_kvs_response;
typedef struct dyad_kvs_response dyad_kvs_response_t;

// Initialize DYAD and create a "dyad_ctx_t" object
int dyad_init(bool debug, bool check, bool shared_storage,
        unsigned int key_depth, unsigned int key_bins, 
        const char *kvs_namespace, const char* prod_managed_path,
        const char *cons_managed_path, bool intercept, dyad_ctx_t **ctx);

// Save (or "commit") file metadata into the Flux KVS
int dyad_commit(dyad_ctx_t *ctx, const char *fname);

// Fetch file metadata from the Flux KVS
int dyad_fetch(dyad_ctx_t *ctx, const char* fname, 
        dyad_kvs_response_t **resp);

// Initiate a Flux RPC call and retrieve (or "pull") the file from the producer
int dyad_pull(dyad_ctx_t *ctx, const char* fname,
        dyad_kvs_response_t *kvs_data);

// Free the dyadd_kvs_response_t objects created by "dyad_fetch"
int dyad_free_kvs_response(dyad_kvs_response_t *resp);

// Finalize DYAD and deallocate the "dyad_ctx_t" object
int dyad_finalize(dyad_ctx_t *ctx);

The library also provides two convenience functions for producing and consuming data:

// Wraps "dyad_commit"
int dyad_produce(dyad_ctx_t *ctx, const char *fname);
// Wraps "dyad_fetch" and "dyad_pull"
int dyad_consume(dyad_ctx_t *ctx, const char *fname);

Besides creating this library, this PR updates the C "sync" API and the C++ "stream" API to use the new "core" library.

Currently, to use the "core" library, the Automake files do the following:

• Link the API libraries against libdyad_core.la using libtool
  • When doing this, libtool automatically sets the RPATH of the API libraries to the path stored in the Autoconf variable libdir
• Install libdyad_core.so into libdir

This should mean that user code can use the API libraries without needing to manually link against libdyad_core.so. Alternatively, libdyad_core.la can be made a noinst target. However, this would prevent certain future decisions in the API libraries, such as making the C++ API a header-only library.

Finally, this PR makes some minor reorganization changes to the repo to make it easier for new developers or people outside the project to understand where code is. More specifically, it makes the following changes:

• Renames common to utils
• Renames lib to stream
• Moves the URPC library into its own urpc directory

[ilumsden]

@JaeseungYeom I've fixed most of the comments you've made. The remaining things that are not yet addressed are:

• This comment: "I think there is a bug in consumer code regarding dyad_pull() not being skipped when it should be."
  • Where is this bug? I'm not entirely sure what you are referring to here.
• Stuff about __restrict__
  • I responded to the relevant comment above asking about whether I should use __restrict__ (compiler extension) or restrict (C99 keyword)
• Style fixes
  • I intend to fix this using clang-format once everything is working
• Renaming the error code types and macros
  • I was trying to make something similar to what UCX does. To that end, maybe a type name like dyad_status_t would be better? If you're curious what I'm specifically looking at, the relevant UCX code is here
• Figuring out which pointers can be converted to const
• Function inlining Addressed since posting this message
  • Since I'm not positive if inline is compatible with PerfFlow Aspect (PFA), I think I'll manually inline (with the inline keyword) when PFA is not enabled.

I've resolved all the comments above that I have either fixed/addressed. The only ones that are still open are ones that either haven't been addressed or ones that I have responded to.

[ilumsden]

@JaeseungYeom this PR is almost done. The only comment you made above that I haven't yet resolved is the comment about renaming the error/return codes and the associated type. Otherwise, everything you've mentioned above has been resolved. I have also tested this PR using the ECP demo workflow, and it is working as intended.

At this point, I am going to open the PR as "ready for review". We can discuss the return code naming later this week. Then, all that will be left is any final reviewing you want to do.