Our paper is public at: https://arxiv.org/pdf/2012.15409.pdf

The repo of UNIMO has been moved to https://github.com/PaddlePaddle/Research/tree/master/NLP/UNIMO

The paper proposes a unified-modal pre-training architecture, namely UNIMO, which can effectively adapt to both single-modal and multi-modal understanding and generation tasks.