-
### Question
Does anyone have carried out the pretraining with Mixtral 8×7B? When I run the petraining script, one problem occured like the figure shown below. I just add a llava_mixtral.py to the ll…
-
can you give the script about contnue pretraining?
-
The current ckpt seems only contains encoder. Is it possible to upload the full checkpoint for the user to continue the fine-tuning? Thanks for the great work!
-
Hi,
My team and I are trying to duplicate the results of your paper, but cannot. Would it be possible to gain access to the code that pretrains the data? That would help us a lot. Thank you.
-
## 🐛 Bug
> 微调会自动删除ep文件, 导致微调结束后找不到需要ep文件
#### Code sample
训练参数
```
torchrun \
--nnodes 1 \
--node_rank 0 \
--nproc_per_node ${gpu_num} \
../../../funasr/bin/train.py \
++model="${model_nam…
-
Hello! I'm running EEG pretraining with code stageA1_eeg_pretrain.py. I'm wondering do you provide the Pretraining EEG Dataset? I saw the path in this code (../dreamdiffusion/datasets/mne_data/) doesn…
-
we are still waiting for pretraining script, we don't want to download datasets. (it's the whole point of pretraining).
we need a demo to test with our own custom images/data.
Thank you
-
I followed the instruction to setup the environment and run pretraining of the T5 baseline. I am using 4x A800 GPUs on 2 nodes so I tweaked the script a little bit. Here is the command I am using:
…
-
I think there could be value in creating a separate dataset for pretraining. It would cover the same chemical space as the standard SPICE dataset, but have many more conformations and be computed at …
-
Currently unsloth offers a customized version of gradient checkpointing that claims to be better. The only way I'm aware of using it is with the below code.
```
model = FastLanguageModel.get_peft_mo…