Closed namespace-Pt closed 2 years ago
hi @namespace-Pt Sorry for the late answer!
I am not sure I completely got your point. For the FLOPS estimation, we rely on the derivation from Minimizing FLOPs to Learn Efficient Sparse Representations. The probabilities are directly estimated from the length of the posting lists (for both documents and queries, where for the later we simply "index" them).
let us know if you need more details, Thibault
Thank you @thibault-formal! I'll read the paper.
BTW, do you have a checkpoint of SPLADEv2-distill
? I tried to reproduce your result but failed. I found that using the distillation we donot need a ground-truth passage to each query, so did you use the queries that are absent from qrels.train.tsv
to train the student model?
I would appreciate it if you could also provide the SPLADEv2-max
(MRR@10=0.34) checkpoint. I want to see how the model learns to distribute the tokens.
Hi @namespace-Pt ,
@sclincha @namespace-Pt the weights for SPLADEv2-max
can actually be found in the weights
folder in this repo.
@sclincha @thibault-formal Thank you! I'll check it out.
I recently read your SPLADE paper and I think it's quite interesting. I have a question concerning FLOPs calculation in the paper.
I think computing FLOPs for an inverted index involves the length of the activated posting lists(the overlapping terms in query and document). For example, a query
a b c
and a documentc a e
, since we must inspect the posting list of the overlapping termsa
andc
, the flops should at least bebecause we perform summation for each entry in the posting list. However, in the paper you compute FLOPs by the probability that
a
,b
,c
are activated in the query andc
,a
,e
are activated in the document. I think this may underestimate the flops of SPLADE because the less sparse the document, the longer posting lists in the inverted index.