search

IBM / federated-learning-lib

A library for federated learning (a distributed machine learning process) in an enterprise environment.
Other
500 stars 137 forks source link

Problem with printing graphics with sklearn_logclassification #50

Closed Enrique-Marmol closed 3 years ago

Enrique-Marmol commented 3 years ago

Hello everyone, I was running the sklearn_logclassification model with a dataset of mine. The model runned successfully, but when I wanted to plot the graphics as the keras clasiffier the variables eval_party_accuracy and iterations are empty. imagen imagen

I named agg_config as agg, and as in sklearn_logclassification there is not epochs so I wrote max iter. I tried to seek the problem, but I could not find anythig. How do I have to change the code to make it work for a sklearn_logclassification model? Thanks in advance

shashank215r commented 3 years ago

@Enrique-Marmol The approach you used here is correct, to confirm if the configuration changes are on point for this approach, Could you paste the hyper parameters you are using for this training run(Just the hyperparams section of the config). Also try printing eval_party_accuracy and iterations at the start of cell 8 to see what is being saved every round.

Enrique-Marmol commented 3 years ago

These are my hyperparams:

hyperparams:
  global:
    parties: 2
    rounds: 3
    termination_accuracy: 0.9
  local:
    training:
      max_iter: 2

When I printed eval_party_accuracy and iterations they are empty, [[],[]]. So I decided to debug the program, and what I could find is that the variable that save the metrics, in sklearn model is empty, but in keras are filled. I mean, in keras we have imagen However, in sklearn I have imagen This is what I could achieve at the moment.

shashank215r commented 3 years ago

@Enrique-Marmol Could you share the notebooks you are using for this test, the approach seems to be fine here and it should give you the evaluations on aggregator side. Let me check your notebook files and see if there is anything fishy going on there.

To speed things up, how about we continue this discussion in https://ibm-fl.slack.com/ ?

shashank215r commented 3 years ago

Seems like parties are running in private mode. Adding the below line in party notebook after party registration would fix the issue.


party.proto_handler.is_private = False  ## allows sharing of metrics with aggregator