Closed KKNakkav2 closed 5 months ago
Can authors please tell if this issue is expected? or is it related to wrong setting in the configuration file. Thank you
In the current version, the multi-GPU training is not supported with offsite-tuning. Thank you!
Thank you for letting me know.
Dear @rayrayraykk,
I'm trying to run federated
offsite tuning
code in the multi-gpu setting by setting the parameterfederate.process_num
When I set the value of
federate.process_num
to >=2 in our server with 4 GPUs, I encountered an issue inClient.py
withinOffsiteTuningClient
class. The error is as follows from :It looks the
self._model
is deleted at https://github.com/alibaba/FederatedScope/blob/7f086944c57f85c7594bde44d4f6b981f0de6845/federatedscope/llm/offsite_tuning/client.py#L37 within theOffsiteTuningClient
class.Can you please advise how to overcome this issue. Any pointers would also help me to fix the issue myself. Thanks a lot.