Paper plan

[akusok]

The paper goes approximately like this:

Introduction

• [x] Introduction to federated learning and ELM

Experimental setup for federated learning in the paper

• [x] Use case that shows why we need federated learning, and shows the paper setup
• [x] House pricing dataset, separate into users with different amount of data
• [x] Show that a lot of data gives good results, little data gives bad results

Present our implementation of federated ELM

• [x] 1) Start from ELM setup: number of neurons, weights, biases. Save on disk.
• [ ] 2) Transform user data into batches of (H'H, H'Y) in chunks (maybe 100 samples at a time)
• [ ] 3) Train an ELM on these data batches. Data can come from any user, no need to have the same user.
• [ ] 4) Send back output weights to users
• [ ] 5) Evaluate ELM performance with these output weights

New stuff in the paper

• [x] Hiding data 1: Add noise to batches of (H'H, H'Y) and see how ELM performs
• [x] Hiding data 2: Add label noise to data, see how ELM performs
• [ ] Hiding data 3: Try differential privacy idea: generate random data samples, see how ELM performs

[akusok]

In Federated methods, we will never see data (X, Y) directly. Instead we will use (H'H, H'Y) for training.

All code for each user is in a separate function (or separate code file). Only that code can read data (X,Y) for that user. Users share out the processed data (H'H, H'Y).

1. Generate weights, biases etc and share for all users.
2. Get data chunks (H'H, H'Y) from each client.
3. Send all data chunks (H'H, H'Y) to every client.
4. Finish training ELM from (H'H, H'Y) and evaluate client performance. Finish training is very easy: add up all H'H together, H'Y together, then B = solve(H'H_sum, H'Y_sum)

code example

W, B = generate_weights_and_biases()
user1_data = user_1_function(W, B)

def user_1_function(W, B):
  x1, y1 = load_data()  # only exists inside this function
  H1 = ...
  H'H = H1.T.dot(H1)
  ...
  return (H'H, H'Y)  

[akusok]

[tamiratGit]

Hi, Anton. Sorry, I am confused now, Let sat the server call "user_1_function(W, B)" which returns (H'H, H'Y). Can the server call "user_1_function(W, B)" with an aggregate of H'H, H'Y (of C1, C2, C3). I couldn't figure out how the FL server managed the process.

[akusok]

You should have another function for that, like "compute_accuracy(HH, HY)".

[akusok]

Or you can make a user an object, that stores data of W, bias, HH_own, HY_own, [HH of other users], [HY of other users]. Then give that object functions like "get_data(W, bias) -> HH_own, HY_own", and "add_other_users_data(HH, HY)", "compute_accuracy()". You can create data structures and objects that you need, din't limit yourself to what we discussed.