accelerate VC lookup when using FI_AV_TABLE addressing

[hppritcha]

Currently, the gnix_vc objects associated with a gnix_fid_ep instance are always stored in a hash table. For each send/recv/rma/atomic operations posted to aFI_EP_RDM endpoint, the _gnix_ep_get_vc method must be invoked to lookup the VC associated with the target address.

For the case of FI_AV_TABLE., where the application is essentially providing an index in to a table of addresses, it would be more efficient to use a vector for storing pointers to the VC's. This would allow a bypass of much of the logic in the first part of __gnix_ep_rdm_get_vc. Rather than first checking with _gnix_av_lookup, creating the hash key from the av_entry, then doing a lookup in the hash table, the code could be more like

   if (av->type == FI_AV_TABLE) {
        if (ep->vc_table[dest_addr] == NULL) {
                /* code pretty much like exists to alloc/connect vc */
                ep->vc_table[dest_addr] = vc;
        } else 
               *vc_ptr = ep->vc_table[dest_addr];

There's already a hook in the gnix_fid_ep for using a vc_table rather than a vc_ht.

This should help PGAS models like SHMEM, etc (which tend to use FI_AV_TABLE) small RMA puts, etc. a lot.

[e-harvey]

Should the size of gnix_fid_ep->vc_table be able to grow? Should the initial size of the table be set using __gnix_dom_ops_get_val with the GNI_VC_ID_TABLE_CAPACITY flag?

[hppritcha]

Hi Evan

I would expect that 95% of use cases will involve a single one time insert of entries in to the table. However since the code in gnix_av.c already supports grow able table is say it would be good to do something similar for this optimization.

Howard

Am 12.02.2016 um 13:16 schrieb Evan Harvey notifications@github.com:

Should the size of gnix_fid_ep->vc_table be able to grow? Should the initial size of the table be set using __gnix_dom_ops_get_val with the GNI_VC_ID_TABLE_CAPACITY flag?

— Reply to this email directly or view it on GitHub.