search

42Shawn / LLaVA-PruMerge

LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models
Apache License 2.0
85 stars 4 forks source link

RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 16 but got size 1 for tensor number 1 in the list. #11

Open taokz opened 2 months ago

taokz commented 2 months ago

@42Shawn @mu-cai Hi LLaVA-PruMerge Team, thank you for open-sourcing your awesome work. I tried to run fine-tuning script (for both llava v1.0 and v1.5), but I encountered the mismatch error as shown in the title. Could you provide some hints for it. I appreciate your feedback in advanced.

taokz commented 2 months ago

The error was happened when I used: image_features = self.token_prune_merge_advanced_plus(images, if_adaptive=True, reduction_ratio=1/8)

and I can train the model successfully using image_features = self.token_prune_merge_advanced(images, if_adaptive=True, reduction_ratio=1/8)

hasanar1f commented 2 months ago

Same here. Is token_prune_merge_advanced_plus compatible with different reduction ratios (when adaptive is False) yet? I think not.