UniFed: All-In-One Federated Learning Platform to Unify Open-Source Frameworks

🌟For the benchmark result📊, please check our website.👈👈👈

Evaluation Scenarios

Cross-device horizontal

Cross-silo horizontal

Cross-silo vertical

Installation

Requirements

• Python
  Recommend to use Python 3.9. It should also work on Python>=3.6, feel free to contact us if you encounter any problems.
  You can set up a sandboxed python environment by conda easily: conda create -n flbenchmark python=3.9
• Command line tools: git, wget, unzip
• Docker Engine
• NVIDIA Container Toolkit if you want to use GPU.

Install From PyPI

pip install flbenchmark colink

Launch a benchmark (auto-deployment on AWS with a controller)

We highly recommend to use cloud servers(e.g. AWS) to run the benchmark. Here we provide an auto-deploy script on AWS.

Set up the controller server

• Set up a web server(e.g. Apache2, Nginx, PHP built-in web server) with PHP support with files on controller folder.
• Change CONTROLLER_URL="http://172.31.2.2" in install_auto.sh to the controller's web url.

Launch servers

• Launch enough servers you need for the benchmark(e.g. 179 servers for femnist) on AWS and set the user data to the following script. (remember to replace the http://172.31.2.2 with the controller's web url)

  #!/bin/bash
  sudo -i -u ubuntu << EOF
  echo "Hello World" > testfile.txt
  wget http://172.31.2.2/install_auto.sh
  bash install_auto.sh > user_data.out
  EOF

• Wait for server setups, when they are finished you could see all servers' ip under controller/servers.
• Get server list via python get_server_list.py, and copy the server_list.json to exp folder.

Set up the framework operator

• Change the working directory to exp for later steps.
• Register users via python register_users.py.
• Launch corresponding framework operators ./start_po.a <unifed_protocol_name> <repo_url>. For example, ./start_po.a unifed.flower https://github.com/CoLearn-Dev/colink-unifed-flower.git.

Start an evaluation

• Generate the exp/config_auto.json from wizard.
• Start an evaluation via python run_task_auto.py

Launch a benchmark (manual deployment)

Alternatively, you could also manually deploy on your own cluster. You need to prepare enough Ubuntu 20.04 LTS servers based on the needs of the benchmark, and you should set up the environment on these servers. Here we provide one script to set up the environment.

Set up servers

• Download install.sh on home directory.
• Change SERVER_IP="172.16.1.1" in install.sh to corresponding server ip.
• Execute install.sh.
• Record all servers' ip and ~/server/host_token.txt in the following json format.

  {
  "test-0": [
      "http://172.31.4.48:80",
      "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJwcml2aWxlZ2UiOiJob3N0IiwidXNlcl9pZCI6IjAyMGJhNzkyNzk0ZTlmMWUwZWZmNTEyOGM4NDdjZmE0MmRlNTllY2I1ODM4MzU4MDBmN2QwMzM1Yzg2YWFjZTViOSIsImV4cCI6MTY4OTM0ODAyM30.UP5JUYdbL-MkZDTSVuBHnIHoun1VkfcRgsBLV119v6A"
  ],
  "test-1": [
      "http://172.31.15.143:80",
      "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJwcml2aWxlZ2UiOiJob3N0IiwidXNlcl9pZCI6IjAyNzJhNjgxNDg0NDMwNTFmZTI2NDFlYmZiNjM2MDgxODM5YmQ5NDdkZGFhNTcwYjY3MjU0MTI2NjU5YzBmZjVjYSIsImV4cCI6MTY4OTM0ODAzNH0.2HHQYzcMjif0ZkhSyltlOSC1ydsgWS8H_no5wWvohw0"
  ]
  }

Set up the framework operator

• Change the working directory to exp for later steps.
• Put the json file about servers from the last step to exp/server_list.json.
• Register users via python register_users.py and record the user id for later use.
• Launch corresponding framework operators ./start_po.a <unifed_protocol_name> <repo_url>. For example, ./start_po.a unifed.crypten https://github.com/CoLearn-Dev/colink-unifed-crypten.git.

Start an evaluation

• Generate the exp/config.json from wizard. Remember to replace the user_id with the user id you got from the previous steps.
• Start an evaluation via python run_task.py

License

This project is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.