Required time for training

hustvl / Senna

Bridging Large Vision-Language Models and End-to-End Autonomous Driving

Apache License 2.0

168 stars 5 forks source link

Required time for training #3

Open kemaloksuz opened 1 week ago

kemaloksuz commented 1 week ago

Hi, thanks for the great work. Could you please specify the time and compute requirements to train Senna?

rb93dett commented 1 week ago

We train Senna on 8xA800 GPUs (80G), with a training duration of approximately 40 hours using 800k QA samples. By reducing the batch size, training on GPUs with 40G or 24G of memory should be feasible.

kemaloksuz commented 1 week ago

Thanks for the response. Does this include Senna E2E part or only Senna VLM? If not, how long does it take to train SennaE2E in addition to Senna VLM?

rb93dett commented 1 week ago

Senna-E2E and Senna-VLM can be trained independently in parallel. The training time for Senna-E2E depends on the training data and model used. With VAD as the base model on the nuScenes dataset, the training duration is approximately 12 hours (on 8 * 4090 GPUs).

kemaloksuz commented 6 days ago

Thanks for the answer. I think the details will be more clear once you release the code. Before then, can I ask 2 more questions please:

Don't you need to train Senna-VLM first, and then train Senna-E2E as Senna E2E will be using the representation of Senna VLM? This was my understanding from the paper, but might be missing something as you mentioned that they can be trained in parallel.
Also, how many epochs do you train VAD on nuScenes? Normally, it is trained for 60 epochs, in which case I'd expect to take more than 12 hours on 8 GPUs.