Thanks for your work.I have a question. Since the prompt in the model's input already describes the content represented by the vectors, why is it necessary to align the vectors with the LLM during the pre-training process? Are the vectors used to help the model further understand the driving scenario based on the prompt? Are the labels in the pre-training process the prompts generated by lanGen? What is the purpose of the 100k question-answer pairs in the pre-training process?

wayveai / Driving-with-LLMs

PyTorch implementation for the paper "Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving"

Apache License 2.0

408 stars 38 forks source link

Open lilpeng opened 3 months ago

lilpeng commented 3 months ago

Could you explain what the model's input and the corresponding labels are during the pre-training stage to help me better understand this process?

MLgeek7325 commented 3 months ago

您好，您的邮件我已收到！