LatticeX-Foundation / Rosetta

A Privacy-Preserving Framework Based on TensorFlow
GNU Lesser General Public License v3.0
566 stars 110 forks source link

[Mystique] 测试基于zk协议的操作,编译报错 #124

Open Jiahu235 opened 1 year ago

Jiahu235 commented 1 year ago

rosetta中没有zk(mystique)协议的测试文件,我们想测试基于zk(mystique)协议的操作,例如relu操作 测试文件编译时报错:

/usr/bin/ld: ../../../../lib/libzkp-mystique.so: undefined reference to CheatRecord::put(std::string const&)
/usr/bin/ld: ../../../../lib/libzkp-mystique.so: undefined reference to CheatRecord::message
collect2: error: ld returned 1 exit status
make[2]: *** [modules/protocol/zk/mystique/CMakeFiles/relu-zk.dir/build.make:130:bin/relu-zk] 
make[1]: *** [CMakeFiles/Makefile2:604:modules/protocol/zk/mystique/CMakeFiles/relu-zk.dir/all] 
make: *** [Makefile:136:all] 
cpp make install Elapsed Time (using $SECONDS): 4 seconds

首先,测试文件中并没有调用与CheatRecord相关的内容;其次,'CheatRecord::put(std::string const&)' 和'CheatRecord::message' 在'emp-zk-bool.cpp'文件中有定义。报错的可能原因是什么?

Jiahu235 commented 1 year ago

为了便于排错,我补充了我的相关操作:

#include <string>
#include <unordered_map>
#include <vector>
#include <iostream>
#include <algorithm>

#include "cc/modules/protocol/utility/include/_test_common.h"
#include "cc/modules/protocol/utility/include/version_compat_utils.h"

#include "cc/modules/protocol/zk/mystique/include/mystique_impl.h"

#include "cc/modules/protocol/zk/mystique/include/mystique_ops_impl.h"
#include "cc/modules/protocol/utility/include/util.h"

#include "cc/modules/protocol/zk/mystique/include/wvr_util.h"
#include "cc/modules/protocol/zk/mystique/include/mystique_internal.h"

using namespace emp;
using namespace rosetta;
using namespace std;

int party = 0;
int port = 48000;
void *wvr_impl;
std::string node_id;
std::string config_json;

std::string task_id = "1";
std::string msgID = "1";
std::string protocol_name("mystique");

rosetta::MystiqueProtocol* myProt;
rosetta::MystiqueProtocolFactory* myProtFact;

attr_type info = {
  {"m", 0}, {"k", 0}, {"n", 0}, {"transpose_a", 0}, {"transpose_b", 0}, {"rh_is_const", 0}
};
attr_type* attr_info = &info;
msg_id_t msg_id(msgID.c_str(), msgID.size());
shared_ptr<ProtocolContext> context = myProt->GetMpcContext();
rosetta::MystiqueOpsImpl* myOpImpl = new rosetta::MystiqueOpsImpl(msg_id, context);

ZkIntFp ZkZero = ZkIntFp(int64_t(0), party);

void Reveal(std::vector<ZkIntFp>& inputArr, std::vector<int64_t>& expected){
  int64_t size = inputArr.size();

  // for (int64_t i = 0; i < size; i++){
  //   inputArr[i].reveal(expected[i]);
  // }

  ZkIntFp* input = new ZkIntFp[size];
  for (int64_t i = 0; i < size; i++) input[i] = inputArr[i];
  ZkIntFp tmp;
  tmp.batch_reveal(input, size);
}

void test_Relu()
{
    int64_t size = 1*32*32*16;
    std::vector<ZkIntFp> input(size, ZkIntFp(int64_t(-1), party));
    vector<ZkIntFp> output(size, ZkIntFp(int64_t(1), party));
    vector<int64_t> expected(size, 0);

    std::vector<string> inputArr(size);
    std::vector<string> outputArr(size);

    convert_mac_to_string(input, inputArr, true, 1);

    myOpImpl->Relu(inputArr, outputArr, attr_info);

    convert_string_to_mac(outputArr, output, true, wvr_impl);

    Reveal(output, expected);
}

int main(int argc, char** argv)
{
  emp::parse_party_and_port(argv, &party, &port);
  string logfile = "cc/modules/protocol/zk/mystique/log/relu-" + to_string(party);
  Logger::Get().log_to_stdout(false);
  Logger::Get().set_filename(logfile + "-backend.log");
  Logger::Get().set_level(0);
  rosetta_old_conf_parse(node_id, config_json, party, "cc/conf/zk-CONFIG.json");
  myProt = new rosetta::MystiqueProtocol(task_id);
  IOManager::Instance()->CreateChannel(task_id, node_id, config_json);
  myProt->Init(logfile + "-console.log");
  shared_ptr<NET_IO> net_io = myProt->GetNetHandler();
  string node_id_0 = net_io->GetNodeId(0);
  string node_id_1 = net_io->GetNodeId(1);
  vector<string> reveal_receivers = {"P0", "P1"};
  rosetta::attr_type reveal_attr;
  reveal_attr["receive_parties"] = receiver_parties_pack(reveal_receivers);

  test_Relu();

  myProt->Uninit();

  return 0;
}
function(compile_ex_zkp category)

file(GLOB EXAMPLE_SOURCE_FILES RELATIVE "${CMAKE_CURRENT_SOURCE_DIR}/${category}" "${category}/*.cpp")

  foreach(EXAMPLE_SOURCE_FILE ${EXAMPLE_SOURCE_FILES})

    string(REGEX REPLACE "(.*)\\.cpp" "\\1" EXAMPLE_NAME ${EXAMPLE_SOURCE_FILE})

    set(EXAMPLE_TARGET "protocol_zkp_mystique_${category}_${EXAMPLE_NAME}")

    add_executable(${EXAMPLE_TARGET} ${category}/${EXAMPLE_SOURCE_FILE})

    target_link_libraries(${EXAMPLE_TARGET} zkp-mystique protocol-utility ${EMP-TOOL_LIBRARIES} ${EMP-OT_LIBRARIES} ${EMP-ZK_LIBRARIES})

  endforeach()

endfunction()

compile_ex_zkp(tests)
Jiahu235 commented 1 year ago

编写测试文件时遇到的其他问题

1.msg_id_tProtocolContext类的使用

msg_id_t的作用,取值范围以及含义是什么? ProtocolContext中相关参数的作用,取值的要求以及含义是什么?

2. 测试前的初始化

rosetta中没有zk(mystique)协议的测试模板,因此我只能仿照mpc协议的测试模板 mpc协议测试模板的初始化操作如下:

#define PROTOCOL_MPC_TEST_INIT(partyid)                                                           \
  GET_PROTOCOL(mpc_proto);                                                                        \
  string logfile = "log/mpc_tests_" + protocol_name + "_" + get_file_name(__FILENAME__) + "-" + \
    to_string(partyid);                                                                           \
  Logger::Get().log_to_stdout(false);                                                             \
  Logger::Get().set_filename(logfile + "-backend.log");                                           \
  Logger::Get().set_level(0);                                           \
  string node_id;                                                                     \
  string task_id = "";\
  string config_json;                                                                 \
  rosetta_old_conf_parse(node_id, config_json, partyid, "CONFIG.json");               \
  IOManager::Instance()->CreateChannel(task_id, node_id, config_json);                         \
  mpc_proto->Init(logfile + "-console.log");                              \
  shared_ptr<NET_IO> net_io = mpc_proto->GetNetHandler();    \
  string node_id_0 = net_io->GetNodeId(0);                                            \
  string node_id_1 = net_io->GetNodeId(1);                                            \
  string node_id_2 = net_io->GetNodeId(2);                                            \
  vector<string> reveal_receivers = {"P0", "P1", "P2"};                                      \
  rosetta::attr_type reveal_attr;                                                     \
  reveal_attr["receive_parties"] = receiver_parties_pack(reveal_receivers);

这段代码中:

仿照这段代码对zk协议进行初始化时,有什么需要注意的?

joyoFeng commented 1 year ago
  1. task_id主要用于多任务执行(单任务直接空即可)
  2. CreateChannel是构建网络监听和连接,每个net_io标识一个连接,一般关联一个节点id
  3. reveal_attr用来指明reveal secret到那个接收节点,(这里有内部的一些适配,外部reveal_receivers跟内部可能不一致需要转换)
  4. ROTOCOL_MPC_INTERNAL_TEST_INIT只是方便测试使用宏,做一些初始化操作。(注意zk应该是2pc的,应对应适配)
joyoFeng commented 1 year ago

rosetta中没有zk(mystique)协议的测试文件,我们想测试基于zk(mystique)协议的操作,例如relu操作 测试文件编译时报错:

/usr/bin/ld: ../../../../lib/libzkp-mystique.so: undefined reference to CheatRecord::put(std::string const&)
/usr/bin/ld: ../../../../lib/libzkp-mystique.so: undefined reference to CheatRecord::message
collect2: error: ld returned 1 exit status
make[2]: *** [modules/protocol/zk/mystique/CMakeFiles/relu-zk.dir/build.make:130:bin/relu-zk] 
make[1]: *** [CMakeFiles/Makefile2:604:modules/protocol/zk/mystique/CMakeFiles/relu-zk.dir/all] 
make: *** [Makefile:136:all] 
cpp make install Elapsed Time (using $SECONDS): 4 seconds

首先,测试文件中并没有调用与CheatRecord相关的内容;其次,'CheatRecord::put(std::string const&)' 和'CheatRecord::message' 在'emp-zk-bool.cpp'文件中有定义。报错的可能原因是什么?

请首先查看是否所有的GitHub的submodules已经下载,然后重新清理build

Jiahu235 commented 1 year ago

我重新确认过了,所有的submodules都已经下载,清理过build之后重新编译,仍然报相同的错误

wercsrsr commented 1 year ago

作者你好,我目前也在研究Mystique协议并复现。现在已经把ZK部分编译通过了。后续想进行Mystique相关函数的测试。测试代码如上图所述:https://github.com/LatticeX-Foundation/Rosetta/issues/124#issuecomment-1558795001。 编译该测试函数后,已经成功生成了相应的二进制文件。 那么,我应该如何设置命令行参数来运行这个可执行文件呢?

wercsrsr commented 1 year ago

如果直接运行该relu.cpp测试文件,会出现以下错误 image