-
1、train_dpo.py需要from data_processing.utils import load_jsonl, load_json，缺失data_processing文件
2、modality_lengths函数中要计算answer字段的长度，dpo数据集构造中没有answer字段
3、A800 80G显卡训练显存不够，如何优化
命令如下：
torchrun --nproc_p…
-
您好，我在下载了您的项目之后，修改了minigpt4_vicuna0.yaml和/mnt/sda1/mateng/BAP-Jailbreak-Vision-Language-Models-via-Bi-Modal-Adversarial-Prompt/MiniGPT-4/minigpt4/configs/models/minigpt4_vicuna0.yaml里面的模型路径，但是执行VAP.py的…
-
### 🚀 The feature
Add support for vision-language models like CLIP or LIT.
### Motivation, pitch
Dear torchvision team,
I am sorry if I missed discussions about this or a specific reason why you h…
-
qwen2-vl has always been memory hungry (compared to the other vision models) and even with unsloth it still OOMs when the largest llama3.2 11b works fine.
I'm using a dataset that has high resolution…
-
Hi,
I am trying to provision a serverless endpoint using azapi_resource for the azureml://registries/azureml/models/Phi-3.5-vision-instruct model in eastus2 but i am getting an error that "The reques…
-
thanks for the great work. I was trying to reproduce your code, I noticed during pretraining, if you set the `mm_vision_output_token_count = 576` you will get:
```
File "llava-token-compression/ll…
-
同学你好，非常感谢你对这一系列论文的整理和梳理，真的帮助很大！在阅读文献时注意到，仓库中部分标注为“2024-NeurIPS”的论文是“2023-NeurIPS”。以下是我发现的相关论文列表，供参考：
2023-NeurIPS：[Enhancing Adversarial Contrastive Learning via Adversarial Invariant Regularizatio…
-
https://medium.com/ubiai-nlp/how-to-fine-tune-llava-on-your-custom-dataset-aca118a90bc3
LLaVA exemplifies the synergy achieved through the convergence of language and vision. At its essence, LLaVA em…
-
Hi,
I'm trying to constrain the generation of my VLMs using this repo; however i can't figure out the way to personalize the pipeline for handling inputs (query+image). Whereas it is documented as …
-
### Model description
Lorax's official supported models does not list any vision model. This is a big gap for a very successful product.
Having lorax a critical component in our tech stack without …