-
## 🐛 Bug
I use the script as follow:
CUDA_VISIBLE_DEVICES="0, 1, 2, 3" metaseq-train --task streaming_language_modeling \
data/pile-test/ \
--num-workers 4 \
--reset-dataloader \
--vo…
-
# Ask a Question
How should be the input when we want to do prediction on batch_size more than 1?
### Question
Explain your question here.
this one:
texts = ["Here is some text to encode :…
-
Hello! I find that in npu-only mode, the output size is set to be 1 but I want to use this simulator to generate tokens more than 1. I use stats.tsv in share-gpt folder as cli_config( it contains two …
-
I tried to quantize (add QDQ layers) the gpt2 model:
```
batch_size=8
with QATCalibrate(method="histogram", percentile=99.999) as qat:
model_q = self.model.cuda()
…
-
## Description
GPT2 tests in tests/test_models.py is mysteriously killed. The was found in the recent [nightly tests](https://github.com/dmlc/gluon-nlp/actions/runs/811741483)(cu102-2.0.0b20210502 an…
-
Hi I am trying to benchmark GPT2-large and experienced RuntimeError: The size of tensor a (1024) must match the size of tensor b (1025) at non-singleton dimension 3.
The inputs should able to accep…
-
Helper to reset session (instead of requiring restarting app)
-
during Running run.py for gpt2-medium i am getting following error.
RuntimeError: [TensorRT-LLM][ERROR] CUDA runtime error in cub::DeviceSegmentedRadixSort::SortPairsDescending(nullptr, cub_temp…
-
**Describe the bug**
The sequence length during training is different than specified, in the configs, I've specified seq-len 50016, which is divisible by the tensor-model-parallel-size=4, however, du…
-
Hi, I am trying out this great framework with a self trained GPT-2.
I wanted to use a custom trained model and the base model as tokenizer.
No matter if I use this approach or solely the base mo…