Inference cost code search task.

[VonNeumann2050]

Hello I saw that the codesearch task had an inference cost of about 7 min for the AdvTest and I was wondering how do you compute that ?

Is it the time you got using a single query on 2x P100 or on multiples queries ?

And is the inference cost similar in https://github.com/microsoft/CodeBERT ? If not why does it take so much time and is there a way to reduce it ?

[VonNeumann2050]

Because on your table

It is unclear whether the inference time are for practising the inference on the whole dataset or just to make one query.

But I guess 7 min is the inference for a whole dataset isn't it ?

[guody5]

7 min is the inference for a whole dataset.