Use interactively without indexing?

[orionw]

Hi there!

I'm looking to use Splade to evaluate just a handful of examples in a programatic way (without reading/writing to disk). The inference_splade.ipynb script was super useful, but I'm looking to evaluate a query over a handful of document rather than only look at the document representations.

Is there an easy way to do this, or will I have to index and write to disk my small number of examples and then retrieve from that?

Thanks!

[thibault-formal]

Hi @orionw If you only have a handful of documents, I guess you can just keep document representations in memory and directly compute dot products (i.e., ranking scores) with query vectors.

[orionw]

Thanks @thibault-formal!

Sorry to ask a potentially naive question, but I want to make sure I am using SPLADE correctly. Is it valid to literally just take the dot product between the two representations or am I missing any important postprocessing steps in SPLADE?

E.g. is this correct to calculate the score by taking the simple dot product?

doc = "this is a test document"
q = "test document"

### from inference_splade.ipynb code ###
import torch
from transformers import AutoModelForMaskedLM, AutoTokenizer
from splade.models.transformer_rep import Splade
model_type_or_dir = "naver/splade-cocondenser-ensembledistil"
model = Splade(model_type_or_dir, agg="max")
model.eval()
tokenizer = AutoTokenizer.from_pretrained(model_type_or_dir)

with torch.no_grad():
    doc_rep = model(d_kwargs=tokenizer(doc, return_tensors="pt"))["d_rep"].squeeze()  
    #### new: compute the query representation ####
    q_rep = model(d_kwargs=tokenizer(q, return_tensors="pt"))["d_rep"].squeeze()  

#### new: compute the score ####
score = torch.dot(doc_rep, q_rep)

Thanks again for your time!

[thibault-formal]

hi @orionw , sorry for the late reply! yes, it's perfectly correct

I close the issue -- feel free to re-open it!

best Thibault