-
请问,Visual Descriptions是怎么通过GPT生成的,如果对于国内用户来说可以用其他的代替吗?
如果想要生成关于OV-COCO数据集相关的视觉描述,应该怎么做?
-
## Description
I am encountering data loading throughput issues while training a large model on Google Cloud Platform (GCP). Here's some context:
I am utilizing Vertex AI pipelines for my training…
-
Hi,
I am looking for a documentation on the list of ports needed for distributed training of XGBoost, specifically, I am interested in XGBoost training with Ray, however, a documentation on other b…
-
Deepspeed support finetune extra model with lora ?
-
### Bug description
I have encountered a hard crash when using a non-scalar pytorch-metric alongside a scalar one and training with a ddp strategy across multiple nodes. This error is extremely fru…
jjhw3 updated
9 months ago
-
The g_loss value in "train_second.py" is nan.
Debugging found that the output value of the model.decoder() function was nan. (line 391, line 402)
There was no problem in train_first.py, but I don't…
-
### Describe the bug
I am using PyTorch DDP (Distributed Data Parallel) to train my model. Since the data is too large to load into memory at once, I am using load_dataset to read the data as an it…
-
This is my first time using WebDataset and I have multiple shards (about 60) with a large number of images. It was working as I would expect in the normal Dataset class when I was using a single GPU. …
-
Hello, I encounted an error when calling `flop_count_table()` in my distributed training code.
The error message is as below. But I checked the input of function `allgather()` and didn't find anythin…
-
我试图用
CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2
这样并行训练,会直接报错
```
ValueError: You can't train a model that has been loaded in 8-bit precision on multiple devices in any distributed mode.…