-
bash training/finetune_RedPajama-INCITE-Chat-3B-v1.sh
My configurations changes as below:
--lr 1e-5 --seq-length 2048 --batch-size 8 --micro-batch-size 1 --gradient-accumulate-step 1 \
--num-layers…
-
Hi,
I'm using multi-node training and I need to know how to calculate the hyperparameter values in the train_redpajama script. Can you please elaborate more on how to set these values?
Here are …
-
```
bash scripts/apptainer_run_quality_signals.sh \
--config configs/rp_v2.0.conf \
--dump_id "2022-49" \
--input_base_uri "file:///path/to/data/root" \
--output_base_uri "file:///path/to…
-
## Issue
Encountered a deadlock while running a JAX-based LLM training script on a TPU-v4-32 pod. SSH'd into worker 0 and ran the script there directly, instead of using `--worker all --command "..."…
-
Can you provide an example of how to launch a training instance? how can one choose the llama model size (350M, 750M, .. 7B, etc)? Thanks in advance
-
- [ ] [SciPhi/AgentSearch-V1 · Datasets at Hugging Face](https://huggingface.co/datasets/SciPhi/AgentSearch-V1)
#### Getting Started
The AgentSearch-V1 dataset is a comprehensive collection of over …
-
Hi :wave:
I was wondering if there is any ongoing incentives for training a new Vicuna model based on the fully open source [OpenLLaMA](https://github.com/openlm-research/open_llama)? This would ul…
-
Hello,
Thank you very much for open-sourcing such an interesting project. I followed the prompt steps and conducted a test. However, there was an error during the rendering process, which caused th…
-
Dear RedPajama team,
I apologize this might not be the right place to ask questions, but I was curious on several aspects of your projects and couldn’t find other better ways to reach out.
I'm a…
zllai updated
8 months ago
-
### 🐛 Describe the bug
运行命令：colossalai run --nproc_per_node 8 --master_port 8822 --hostfile /home/edcuser/models/ColossalAI/examples/language/llama/hostfile.txt --master_addr 192.168.30.3 pretrain…