search

IBM / federated-learning-lib

A library for federated learning (a distributed machine learning process) in an enterprise environment.
Other
500 stars 136 forks source link

How can I use the framework with the adult dataset? #117

Closed deeaoltean closed 1 year ago

deeaoltean commented 2 years ago

I use the KerasFLModel, the adult_dt_data_handler and the KrumFusionHandler. I ran the framework for MNIST and wanted to try it with the adult dataset and this is the error that I get and I am not sure what do I need to modify: 2022-11-19T15:58:24.464Z | 1.0.6 | ERROR | ibmfl.party.party_protocol_handler | Error occurred while performing model.fit Traceback (most recent call last): File "/home/andreia/miniconda3/envs/biciclista/lib/python3.6/site-packages/ibmfl/model/keras_fl_model.py", line 151, in fit_model steps_per_epoch=steps_per_epoch) File "/home/andreia/miniconda3/envs/biciclista/lib/python3.6/site-packages/ibmfl/model/keras_fl_model.py", line 209, in fit_generator training_generator, steps_per_epoch=steps_per_epoch, epochs=epochs) File "/home/andreia/miniconda3/envs/biciclista/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper return func(*args, kwargs) File "/home/andreia/miniconda3/envs/biciclista/lib/python3.6/site-packages/keras/engine/training.py", line 1418, in fit_generator initial_epoch=initial_epoch) File "/home/andreia/miniconda3/envs/biciclista/lib/python3.6/site-packages/keras/engine/training_generator.py", line 181, in fit_generator generator_output = next(output_generator) File "/home/andreia/miniconda3/envs/biciclista/lib/python3.6/site-packages/keras/utils/data_utils.py", line 711, in get six.reraise(sys.exc_info()) File "/home/andreia/miniconda3/envs/biciclista/lib/python3.6/site-packages/six.py", line 719, in reraise raise value File "/home/andreia/miniconda3/envs/biciclista/lib/python3.6/site-packages/keras/utils/data_utils.py", line 687, in get inputs = self.queue.get(block=True).get() File "/home/andreia/miniconda3/envs/biciclista/lib/python3.6/multiprocessing/pool.py", line 644, in get raise self._value File "/home/andreia/miniconda3/envs/biciclista/lib/python3.6/multiprocessing/pool.py", line 119, in worker result = (True, func(args, kwds)) File "/home/andreia/miniconda3/envs/biciclista/lib/python3.6/site-packages/keras/utils/data_utils.py", line 628, in next_sample return six.next(_SHARED_SEQUENCES[uid]) TypeError: 'DataFrame' object is not an iterator

Yi-Zoey commented 2 years ago

Hi @deeaoltean, if you want to use the Adult dataset with Keras model, you will probably need to write your own data handler, the adult_dt_data_handler is for the decision tree model. You can use this tutorial to learn how to create your own data handler.