[Feature]: BumbleBee Code

[deevashwer]

Feature Request Type

Performance

Have you searched existing issues?

Yes

Is your feature request related to a problem?

N/A

Describe features you want to add to SPU

Hi SPU team,

The state-of-the-art Secure 2PC Inference work, titled "BumbleBee: Secure Two-party Inference Framework for Large Transformers" links to SPU for its code. However, I don't see any references to BumbleBee in this repository. Is the code release still in progress? If so, do you have an estimate on when the code will be released?

Describe features you want to add to SPU

N/A

[anakinxc]

@fionser

[fionser]

@deevashwer The matrix multiplication code in the Cheetah back-end is already upgraded to BumbleBee's variant.

• API entry.
• Underlying ciphertext interleaving for compression.
• For the other parts (e.g., multi-segements for approximating Gelu), we can simply drop in the Python hijack in PUMA.
  This might increases the computation overhead a bit.
• However, the Batch Matmul (i.e., used within the Multi-head attention) needs so many dirty hack in the SPU repo. Thus, we are not putting them in the main branch for now.

[deevashwer]

Thanks @fionser for the quick response. Is there a branch that can be used to perform end-to-end benchmarking of the BumbleBee code? It'd be great if there's an end-to-end example for BumbleBee just like the ones for PUMA.

[fionser]

Just simply replace the back-end (from ABY3 to CHEETAH) in the config.json file. It should work. The SPU front-end is designed to be back-end independent. Indeed, the current state of the main branch code should be a bit slower than the reported number in the BumbleBee paper due to the above reasons.

For example, the 2-party config file would like

{
    "id": "colocated.2pc",
    "nodes": {
        "node:0": "127.0.0.1:9320",
        "node:1": "127.0.0.1:9321"
    },
    "devices": {
        "SPU": {
            "kind": "SPU",
            "config": {
                "node_ids": [
                    "node:0",
                    "node:1"
                ],
                "spu_internal_addrs": [
                    "127.0.0.1:9330",
                    "127.0.0.1:9331"
                ],
                "runtime_config": {
                    "protocol": "CHEETAH",
                    "field": "FM64",
                    "enable_pphlo_profile": true,
                    "enable_hal_profile": true,
                    "enable_pphlo_trace": false,
                    "enable_action_trace": false,
                    "experimental_disable_mmul_split": true,
                    "fxp_exp_mode": 0,
                    "fxp_exp_iters": 6
                }
            }
        },
        "P1": {
            "kind": "PYU",
            "config": {
                "node_id": "node:0"
            }
        },
        "P2": {
            "kind": "PYU",
            "config": {
                "node_id": "node:1"
            }
        }
    }
}

[fionser]

@deevashwer Just a "simple" demo on how to add new features on SPU's dispatch workflow https://github.com/secretflow/spu/pull/433

[c-doubley]

@deevashwer 后端的矩阵乘法代码Cheetah已经升级到BumbleBee的变体。

• API 条目。
• 底层密文交错压缩。
• 对于其他部分（例如，用于近似 Gelu 的多段），我们可以简单地在 PUMA 中加入 Python 劫持。 这可能会稍微增加计算开销。
• 然而，批量矩阵乘法（即在多头注意力机制中使用）需要在 SPU 仓库中进行大量脏操作。因此，我们暂时不会将它们放入主分支中。

您好，请问是在头文件中有 // Ref: Lu et al. "BumbleBee: Secure Two-party Inference Framework for Large // Transformers" // https://eprint.iacr.org/2023/1678 的文件才是Bumblebee的相关代码吗？目前这样的代码我只发现了cheetah_dot/cheetah_mul

1. cheetah_dot 中实现了bOLE
2. cheetah_mul 中实现了OLT 不知道我的理解是否正确呢，另外关于GeLU和SoftMax的优化在哪里可以找到呢？