The Herschel Space Observatory provided an unprecedented view of the FIR sky, and has enabled the detection of some of the highest redshift starforming galaxies to date. However, due to it's relatively poor resolution compared to optical data, the SPIRE maps suffer from source confusion. This means it is a non trivial problem to assign SPIRE flux to the appropriate galaxy. This paper builds on the probabilistic framework of XID+, presented in Hurley et al 2017 by using additional prior information from SED templates and photometric redshifts, to robustly assign the infrared flux from SPIRE maps. We present details on the additional SED prior model component. We use XID+SED to show that the brightest Herschel objects are not single objects with massive starformation rates, but can be explained as blends of galaxies with uncontroversial physical properties.