-
Hello,
I'm trying to use Keras Tuner in a distributed tuning fashion.
I did as explained in the docs, and ran the following commands:
```
export KERASTUNER_TUNER_ID="chief"
export KERASTUNER_…
-
If I have a set of julia processes running on different hosts, can I partition the set of packages to cuddle across the cluster?
Will `parcuddle()` achieve linear strong scaling?
-
There is a problem running this query (optimization?) with dask version
```
python 3.12.3 hab00c5b_0_cpython conda-forge
dask 2024.8.1 …
-
I suggest we do this in code, having all MPI ranks on a node figure out what GPUs are available and making sure each rank gets one. This is a simple bit of MPI code that we can write that works generi…
-
First of all, I really appreciated this repo. Thank you very much for the contribution! However, there are 2 functions will not work logically, in distributed.py for the loss and miner wrappers: gath…
-
I meet the following error with training llm on 2 gpu machines:
+ torchrun --nnodes=2 --nproc_per_node=1 --node_rank=0 --master_addr=11.214.25.222 --master_port 9999 --rdzv_id=1986 --rdzv_backend=c10…
-
There are multiple mentions of a multi modal sequence parallel system for inference which can be seamlessly integrated with HF transformers. However, I am not able to follow this through the codebase …
-
I am trying to run single GPU to multinode distributed fine tuning for Llama3-70B and Llama3 8B Models.
Below is my training configuration:
SFT (Llama3 8B & 70B)
Epochs: 3
Gradient Accumulatio…
-
### Feature request / 功能建议
Thanks for open-sourcing such an amazing model and codebase. I would like to ask whether it is possible to open-source the pre-training or fully fine-tune configs for CogV…
-
https://dl.acm.org/doi/10.1145/3600006.3613145