search

opendilab / LMDrive

[CVPR 2024] LMDrive: Closed-Loop End-to-End Driving with Large Language Models
Apache License 2.0
526 stars 48 forks source link

Training process related to misleading information #16

Closed kevinchiu19 closed 5 months ago

kevinchiu19 commented 5 months ago

Hello, thank you for your wonderful work!

I have a question. How is the training setting related to misleading instructions mentioned in the paper implemented? I don’t seem to see it in the code.

Looking forward to your reply, thank you~

deepcs233 commented 5 months ago

Hi!

In the project, we added some labeled misleading data into the training dataset to let the model recognize the misleading instructions. We will release the corresponding data soon!

kevinchiu19 commented 5 months ago

Thanks for your quick reply.

I would like to confirm with you, which part of the code corresponds to "To enable the model to reject misleading instructions, we label the corresponding data as ‘completed’ after the misleading instruction is given for about 1 second." mentioned in the paper.

kevinchiu19 commented 5 months ago

I didn't see the relevant ‘completed’ flag words in the data provided, nor did I see the relevant codes used. I only saw the result['bad_case'] = 'True' flag in parse_misleading.py, but I didn't see the relevant code either.

Is this part of the code not yet release?

deepcs233 commented 5 months ago

Hi! Yes, we haven't released the corresponding code and dataset. However, the current code and dataset can also train an agent that has limited ability to reject misleading instructions. In our paper, the agent rejects the misleading instructions by recognizing the current misleading instruction should be finished in a very short time.

kevinchiu19 commented 5 months ago

Okay, I understand, thank you for your patient reply.