Closed liangdong415 closed 2 weeks ago
The error is likely occurring in the 'search-with-reranking' IPC handler or the reranking function itself. Check the IPC handler in electron/main/database/dbSessionHandlers.ts
and the rerankSearchedEmbeddings
function in electron/main/database/Embeddings.ts
.
IPC Handler: Ensure the search-with-reranking
handler is correctly set up in dbSessionHandlers.ts
.
ipcMain.handle(
"search-with-reranking",
async (
event,
query: string,
limit: number,
filter?: string
): Promise<DBEntry[]> => {
try {
const windowInfo = windowManager.getWindowInfoForContents(event.sender);
if (!windowInfo) {
throw new Error("Window info not found.");
}
const searchResults = await windowInfo.dbTableClient.search(
query,
limit,
filter
);
const rankedResults = await rerankSearchedEmbeddings(
query,
searchResults
);
return rankedResults;
} catch (error) {
console.error("Error searching database:", error);
throw error;
}
}
);
Reranking Function: Verify the rerankSearchedEmbeddings
function in Embeddings.ts
.
export const rerankSearchedEmbeddings = async (query: string, searchResults: DBEntry[]) => {
const { env, AutoModelForSequenceClassification, AutoTokenizer } = await import("@xenova/transformers");
env.cacheDir = path.join(app.getPath("userData"), "models", "reranker");
const tokenizer = await AutoTokenizer.from_pretrained('Xenova/bge-reranker-base');
const model = await AutoModelForSequenceClassification.from_pretrained('Xenova/bge-reranker-base');
const queries = Array(searchResults.length).fill(query);
const inputs = tokenizer(queries, { text_pair: searchResults.map(item => item.content), padding: true, truncation: true });
const scores = await model(inputs);
const resultsWithIndex = searchResults.map((item, index) => {
return {
...item,
score: scores.logits.data[index]
};
});
return resultsWithIndex.sort((a, b) => b.score - a.score).filter(item => item.score > 0);
};
Ensure that the model and tokenizer are correctly imported and initialized.
/electron/preload/index.ts /electron/main/database/Embeddings.ts /electron/main/database/dbSessionHandlers.ts /src/components/Similarity/SimilarFilesSidebar.tsx
Thanks for reporting @liangdong415 we have disabled this feature for now
Describe the bug Error invoking remtoe method 'search-with-reranking' when clicking on "Rerank results"
To Reproduce Steps to reproduce the behavior:
Expected behavior A clear and concise description of what you expected to happen.
Screenshots![image](https://github.com/reorproject/reor/assets/6811528/09798ccc-7cf3-46b4-8c71-96f161a2fdef)
Desktop (please complete the following information):
Additional context Add any other context about the problem here.