Rely on branch predictor to avoid overhead of layer dispatch.

[Kerilk]

I could not measure any overhead on x86-64. As layer become more common, misprediction could become costly.