Describe the solution you'd like
There is a bunch of prior work on splitting postings lists; this allows the high impact list to be traversed (more likely) than the low impact list, and since the low impact lists are typically long, accelerates retrieval. Recent work in EMNLP findings showed that this is highly beneficial to learned sparse retrieval mechanisms. I would like to see if this can be pulled into the PISA engine with perhaps a more elegant design (I currently have a private prototype used in the EMNLP experiments but it's a bit rough for release still). This would then supersede the (currently empty) repo here: https://github.com/JMMackenzie/postings-clipping
Describe the solution you'd like There is a bunch of prior work on splitting postings lists; this allows the high impact list to be traversed (more likely) than the low impact list, and since the low impact lists are typically long, accelerates retrieval. Recent work in EMNLP findings showed that this is highly beneficial to learned sparse retrieval mechanisms. I would like to see if this can be pulled into the PISA engine with perhaps a more elegant design (I currently have a private prototype used in the EMNLP experiments but it's a bit rough for release still). This would then supersede the (currently empty) repo here: https://github.com/JMMackenzie/postings-clipping
Paper: https://aclanthology.org/2022.findings-emnlp.205/
Additional context TBA - This is a draft issue for now.