Luodian
commented
1 year ago That could be possibly issue that you can not train with both single image-text datasets with multiple image-text datasets.
Because the vision_x should be in a tensor shape like (B, T, F, C, H, W), here the T=1 means single images, T=x means multiples in-context images. Same type of datasets should be arranged together so that enabling multi-batch training.
Can you try to load CDG with ic series args.
parser.add_argument(
"--mimicit_ic_path",
type=str,
default="",
help="Path to the new in-context image-text dataset. Should be in format /path/to/xx_instruction.json",
)
parser.add_argument(
"--images_ic_path",
type=str,
default="",
help="Path to the new in-context images dataset. Should be in format /path/to/xx.json",
)
parser.add_argument(
"--train_config_ic_path",
type=str,
default="",
help="Path to the new in-context training config dataset. Should be in format /path/to/xx_train.json",
)Supposedly you could use ic (in-context) to load datasets that could possibly with multiples images as in-context examples.
Darren-greenhand
ZhangYuanhan-AI
I successfully run the training code on each datasets, and I want to train them all with the config:
and it fails , I tried several times and found that once I add CGD dataset, it will come out strange error while it can be trained on its own. the error stack:
I follow the code but I can't understand why it happened as the answer I googled said 'f.read()' is empty given that it works well when single?! π