Open Anonymousplendid opened 2 months ago
We concatenate them if we are using a flat model or if hierarchical model, we will encode independently and pack them in chunks as described in our paper - https://arxiv.org/abs/2404.00596
Understood. Do you have plans to pulic the hierarchical model implementation in the future?
Since facts and laws are segmented sentences, do we just concat all the sentences and use them as a whole text in retrieval?