SparsityPattern::Build::parallel_sync() may be unnecessary?

[roystgnr]

Thanks to @friedmud for pointing this out last night.

I've never seen any noticeable wait in parallel_sync(), but that's probably because I'm running on hundreds of processors, not tens of thousands; he reports that in the latter case it's a huge holdup.

At first glance, it's obvious that we could get a big speedup by doing the same sort of asymptotically-superior redesign that I did in https://github.com/libMesh/libmesh/pull/1600 - 4x on 300 processors would mean at least an order of magnitude speedup on 15K.

At second glance - perhaps rather than computing on active elements only and then doing a parallel sync, we should just have every processor computing on ghost elements as well? I'd guess that's slower on-node, but it's embarrassingly parallel so (until we make some major optimizations in the name of #1617, like perhaps caching "processor neighbors" so we can do that alltoall only once at mesh preparation time) it should be faster on a large enough normal distributed system.

[permcody]

I'm in support of thinking about scaling libMesh to larger processor counts. There's an active SciDac that wants to run large MOOSE(libMesh)-based simulations on 100K cores in the next year or so.

[friedmud]

I'm for this as well. 👍

[roystgnr]

Retitling because #1686 fixed this particular "scales poorly" case; not closing because obviously if we can pull off embarrassingly parallel scaling then that would be a better fix.