How to reuse information from the previous frame？？？

[xiaowuge1201]

How to solve the problem of dimensional misalignment caused by the inconsistency between the number of agents in the previous frame and the number in this frame??? How to cache the data from the previous frame??? If there is relevant code, I hope to open source the relevant code If there is no relevant code, please provide a detailed explanation of the solution to this problem. Thank you very much

[HUXING8]

I have the question same as yours.

[SunHaoOne]

Based on our experimental validation, we have observed some critical insights regarding the usage of caching to ensure consistent inference results: Here's an example of how data is concatenated in the model to maintain alignment:

def forward(self, x_a_past, inputs):
    if x_a is not None:
        x_a = torch.cat([x_a, x_a_past])
    return x_a, x_a[1:]

Model Strategies:

Two model are used to handle cache.

• Full Model: This model does not rely on caching and processes all available data.
• Cache Model: Similar to a KVCache official implementation, this model uses cached results to optimize processing. It's particularly useful when data alignment can be guaranteed.

Strategies for Effective Caching:

• Limiting Agent Count: By restricting the number of agents (e.g., A=8, 16), we focus on the agents that frequently interact with the subject vehicle, simplifying the caching process.
• ID Tracking: If the same IDs are present in consecutive frames, we can align and concatenate them based on their order from the previous frame to utilize cached features effectively.
• Handling ID Changes: In cases where new IDs appear or existing IDs disappear, the full model is deployed to accommodate the change.