I am new to LLM and I get confused by the maximum context length in the long video scenario.
According to the configuration file of long video, the max token is set to 65536 (also mentioned in paper).
Could you please clarify has this number been reached during training? My understanding of context extension (from a couple of simple tutorial) is to make positional embedding extrapolation-able, as such when long sequence is encountered, the model still recognise the positional embedding. However, I wonder what is the average number of tokens when it comes to training with long video and I am curious how much memory it may need to inference with maximum number of token.
Hi,
I am new to LLM and I get confused by the maximum context length in the long video scenario. According to the configuration file of long video, the max token is set to 65536 (also mentioned in paper).
Could you please clarify has this number been reached during training? My understanding of context extension (from a couple of simple tutorial) is to make positional embedding extrapolation-able, as such when long sequence is encountered, the model still recognise the positional embedding. However, I wonder what is the average number of tokens when it comes to training with long video and I am curious how much memory it may need to inference with maximum number of token.