-
**Is your feature request related to a problem? Please describe.**
When the model config has `rampup_batch_size`, we will have model loading errors when the global_batch_size is not set accordingly…
-
**Exact location of issue**
Section 5 "5 Pretraining and Finetuning" of [Longformer Paper](https://arxiv.org/pdf/2004.05150.pdf)
**Problem details**
Section 5 and following sections downstre…
-
I attempted to merge 4 Yi-34B models using the MoE branch of merge-kit (with each token activating 2 experts). These four models are as follows, all of which are based on the Yi34B-base and trained wi…
-
what is the dataset of pre-training model?
I use pre-training model to recognition,but the recognition rate is not good?
what should i do? Can i use a new dataset to re-training or fine tuning on t…
-
hi arthors,
can you open source the pipeline code on pretraining?
appreciate it.!
-
-
作者大大,您好,最近在尝试复现您的论文,遇到一些问题请教一下
1.https://github.com/ViTAE-Transformer/SAMRS/tree/main/Pretraining%20and%20Finetuning 这个链接下的提供的Segmentation Pretrained Models提供的是直接可以用来测试ISPRS Potsdam数据集(我把图裁剪成了512)的吗?…
-
Hello,
I was wondering if it is straightforward to bring older models such as GPT-2 to lit-gpt.
If so, what files/configs do I need to change?
Thank you!
-
详细错误是:
```
session.py 1356 _do_call
return fn(*args)
session.py 1339 _run_fn
self._extend_graph()
session.py 1374 _extend_graph
tf_session.ExtendSession(self._session)
tensorflow.python.…
-
Kindly, help me to solve the problem with this error which I receive when doing the steps to run
python -m model.pretraining.sentiment2017
File "/utils/load_embeddings.py", line 93, in load_word_…