Add caching of UCX endpoints to prevent excessive creation

[ilumsden]

Yet another suggestion from @hariharan-devarajan

We should add code to temporarily cache recently opened UCX endpoints. By doing so, we can minimize how much time we spend opening network connections.

Before starting this, we need to decide a caching strategy. I'd personally suggest LRU as a starting point.

[hariharan-devarajan]

Yes LRU seems a good initial approach. We can look at the literature of networks and see if there are any other better (but simple approaches like LRU) out there. I am tempted to support multiple policies if we see an inherent trade-off between different policies.

[ilumsden]

Initial caching is added