Neighborhood iteration on julia side

[jpfairbanks]

It appears that the biggest performance hit we take in julia is using the gather_successors functiohn https://github.com/stingergraph/stinger/blob/dev/lib/stinger_core/src/stinger.c#L1832

If we could do the iteration over the edge blocks on the julia side, we could avoid a huge overhead. This is the key C macro that this code uses: https://github.com/stingergraph/stinger/blob/dev/lib/stinger_core/inc/stinger_traversal.h#L123.

If the STINGER_FORALL_EDGES_OF_VTX macros were made into julia iterators, we could avoid a lot of overhead. The most direct port is a julia macro with the following usage

@forall_edges_of_vtx begin
    n = STINGER_EDGE_DEST
    w = STINGER_EDGE_WEIGHT
    tf = STINGER_EDGE_TIME_FIRST
    tr = STINGER_EDGE_TIME_RECENT
    t = STINGER_EDGE_TYPE
    if n >= 0 
      size_t where = stinger_size_fetch_add (&kout, 1)
      if where < max_outlen
        out[where] = n
        if(weight) weight[where] = w
        if(timefirst) timefirst[where] = tf
        if(timerecent) timerecent[where] = tr
        if(type) type[where] = t
      end
    end
end

Where the variables are defined in the macro like

/* Use these to access the current edge inside the above macros */
#define STINGER_EDGE_SOURCE source__
#define STINGER_EDGE_TYPE type__
#define STINGER_EDGE_DEST ((current_edge__->neighbor)&(~STINGER_EDGE_DIRECTION_MASK))
#define STINGER_EDGE_DIRECTION ((current_edge__->neighbor)&(STINGER_EDGE_DIRECTION_MASK))
#define STINGER_EDGE_WEIGHT current_edge__->weight
#define STINGER_EDGE_TIME_FIRST current_edge__->timeFirst
#define STINGER_EDGE_TIME_RECENT current_edge__->timeRecent
#define STINGER_IS_OUT_EDGE ((current_edge__->neighbor)&(STINGER_EDGE_DIRECTION_OUT))
#define STINGER_IS_IN_EDGE ((current_edge__->neighbor)&(STINGER_EDGE_DIRECTION_IN))

#define STINGER_GENERIC_FORALL_EDGES_OF_VTX_BEGIN(STINGER_,VTX_,EDGE_FILTER_,EB_FILTER_,PARALLEL_)\
  do {                                                                                                    \
    MAP_STING(STINGER_);                                                                                  \
    struct stinger_eb * ebpool_priv = ebpool->ebpool;                                                     \
    struct stinger_eb *  current_eb__ = ebpool_priv + stinger_vertex_edges_get(vertices, VTX_);           \
    while(current_eb__ != ebpool_priv) {                                                                  \
      int64_t source__ = current_eb__->vertexID;                                                          \
      int64_t type__ = current_eb__->etype;                                                               \
      EB_FILTER_ {                                                                                        \
        PARALLEL_                                                                                         \
        for(uint64_t i__ = 0; i__ < stinger_eb_high(current_eb__); i__++) {                               \
          if(!stinger_eb_is_blank(current_eb__, i__)) {                                                   \
            struct stinger_edge * current_edge__ = current_eb__->edges + i__;                             \
            EDGE_FILTER_ {

#define STINGER_GENERIC_FORALL_EDGES_OF_VTX_END()         \
            } /* end EDGE_FILTER_ */                      \
          } /* end if eb blank */                         \
        } /* end for edges in eb */                       \
      } /* end EB_FILTER_ */                              \
      current_eb__ = ebpool_priv + (current_eb__->next);  \
    } /* end while not last edge */                       \
  } while (0)

This could also get turned into a function that satisfied the do notation interface.

function forall_neighbors(func::Function, s::Stinger, v::Int)
   for u in edgeblocks(s, v)
       f(s, v, u)
   end
end

so that you would use it like

   forall_neighbors(s, v) do s, v, u
       #function body
   end

I guess this requires edgeblocks as an iterator http://docs.julialang.org/en/release-0.5/manual/interfaces/.

I think either way will reduce the overhead and get nearly equal performance.