Closed haoquan24 closed 7 months ago
Hi,
Thanks for reaching out. The adapter's integration is detailed in the adapter module of the codebase. https://github.com/alibaba/FederatedScope/blob/7f086944c57f85c7594bde44d4f6b981f0de6845/federatedscope/llm/model/adapter_builder.py#L147
Traditional FL is sending the full model, and adapter is used to fine-tune a small portion of the LLM's parameters. For full definition and details, you can refer to our paper https://arxiv.org/abs/2309.00363 . And let me know if you have any further questions.
Best regards,
Hi,
Thanks for your time. I have another question. In the yaml file, what does "local_update_steps" represent?
Is the amount of data used for each client training "local_update_steps * batch_size" instead of all?
If not, what data does each client use in one round of training?
Best wishs.
When cfg.train.batch_or_epoch = 'batch'
, the amount of data used for each client training is "local_update_steps batch_size". And When cfg.train.batch_or_epoch = 'epoch'
, the data used are "local_update_steps num_data".
In federatedscope/core/configs/cfg_training.py:
cfg.train.local_update_steps = 1
cfg.train.batch_or_epoch = 'batch'
Hi,
Thanks for the code, I have some questions for you.
When I use fs-llm, the information sent between the client and the server is the adapter. I want to know which part of the code specifically introduces the adapter?
What is the difference between the adapter sent by the client and server and the model sent in general FL?
Thans for your time.