RoBERTa large max input token size issue

[jinmyeongAN]

Question

What if all previous utterances is over than 512 number of tokens?

I know the max input token size of RoBERTa large is 512. [link]

And in your paper, I could find this

context embedding module(CoM) reflects all previous utterances as context.
...
We use an Transformer encoder as a context model (such as RoBERTa).

If you met that kind of problem, then did you use sliding window or something

Thank you

[rungjoo]

As I recall, in the IEMOCAP dataset, there are cases that exceed the max length, so old tokens that make it over the max length will be discarded.

Please refer to the following code. https://github.com/rungjoo/CoMPM/blob/master/utils.py#L76 https://github.com/rungjoo/CoMPM/blob/master/utils.py#L16

[jinmyeongAN]

I understood! Thank you for your answering :)