search

X-LANCE / SLAM-LLM

Speech, Language, Audio, Music Processing with Large Language Model
MIT License
578 stars 52 forks source link

About FSDP，deepspeech. #101

Closed Alex-Songs closed 1 week ago

Alex-Songs commented 5 months ago

Hello， May I know if the current FSDP and DeepSpeech are stable and available for use? Do they support multi-machine multi-card and LORA fine-tuning?

ddlBoJack commented 5 months ago

Sorry that they are not stable. Anyway Welcome PR :)

fclearner commented 3 months ago

fsdp has a bug: sharding_strategy needs “ShardingStrategy” not “str”

rookie0607 commented 3 months ago

fsdp 有一个错误：sharding_strategy 需要“ShardingStrategy”而不是“str”

你解决了吗？ @fclearner

fclearner commented 3 months ago

fsdp 有一个错误：sharding_strategy 需要“ShardingStrategy”而不是“str”

你解决了吗？ @fclearner

yes，try this:https://github.com/wenet-e2e/wenet/blob/24375c51c8ccbf1dac0724f5734a6eae4ac9c428/wenet/utils/train_utils.py#L410

ddlBoJack commented 1 week ago

Refer to this PR #153