alibaba / FederatedScope

An easy-to-use federated learning platform
https://www.federatedscope.io
Apache License 2.0
1.26k stars 206 forks source link

Error when using register_data #748

Closed stupidhhh closed 7 months ago

stupidhhh commented 7 months ago

I followed the steps on the documentation to try to use the MNist dataset, but I get an error when registering the dataset using federatedscope.register.register_data The link to the document is https://federatedscope.io/docs/own-case/#data ; the error is as follows: image

rayrayraykk commented 7 months ago

Hello!

Thank you for bringing this to our attention. It appears that the tutorial could indeed be slightly outdated. To align with the current state of the codebase, the correct implementation of the function should be like this:

def call_my_data(config, client_config=None):
    pass

Please note that if you encounter any issues while implementing the function, feel free to ask for further clarification. Additionally, if you come across any other content in the tutorials that seems out of date, do let us know so that we can update the information accordingly.

Thank you for your contribution!

stupidhhh commented 7 months ago

Thank you for bringing this to our attention. It appears that the tutorial could indeed be slightly outdated. To align with the current state of the codebase, the correct implementation of the function should be like this:

Thank you for your answer, but I have a new problem after changing the code, the new problem is as follows: image The complete code is below:

from torchvision.datasets import MNIST
from torchvision.transforms import transforms

from federatedscope.core.data import BaseDataTranslator
from federatedscope.register import register_data

def load_my_data(config, client_cfgs=None):
    transform = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.1307], std=[0.3081])
    ])
    data_train = MNIST(root='data', train=True, transform=transform, download=True)
    data_test = MNIST(root='data', train=False, transform=transform, download=True)
    translator = BaseDataTranslator(config, client_cfgs)
    fs_data = translator([data_train, [], data_test])
    return fs_data, config

def call_my_data(config, client_config=None):
    pass

register_data("mydata", call_my_data)

I changed the value of data.type in the scripts.example_configs.femnist.yaml file to mydata and ran federatedscope.main.py with this config file.

rayrayraykk commented 7 months ago

The pass in my past answer is code omitted for brevity, and you should implement it like this:

# ... [code omitted for brevity]

def call_my_data(config, client_config=None):
     if config.data.type == "mydata":
         data, modified_config = load_my_data(config)
         return data, modified_config

# ... [code omitted for brevity]