search

zilliztech / feder

Visualize hnsw, faiss and other anns index
392 stars 32 forks source link

feder for text documents #48

Closed safakkbilici closed 2 years ago

safakkbilici commented 2 years ago

I want to visualize my text documents and so i did. However i didn't find any way to get docids in a specific cluster. For example, when I search over a random vector with feder, library gives me a bunch of clusters. Is it possible to obtain which vectors are stored in a specified cluster?

Actually what i exactly wanted and asked is that, in your example you are able to see images with fine search (distance) (first attached file). I want to show small texts instead of images (second attached file). 1 2

alwayslove2013 commented 2 years ago

Currently, we only support displaying images. We will release a new version in this week, which supports showing text, or other user-customized information panels.

const feder = new Feder({
  source: 'faiss',
  filePath: ivfflatIndexFilePath,
  viewParams: {
    mediaType: 'img',
    mediaCallback: () => null,
  },
});

Set mediaCallback to () => null, and hover the node you are interested in, Feder will show the rowID and the clusterID. image

If you want to know which vectors are in a specific cluster, you can try it.

// get the rowIds of vectors that belong to cluster-123.
feder.core.indexMeta.listIds[123] // "list" means "cluster"
safakkbilici commented 2 years ago

thanks for the quick reply, i'm looking forward to the new release! is there any pythonic version of feder.core.indexMeta.listIds[123] code snippet. I think this is for JS.

alwayslove2013 commented 2 years ago

This is the internal data of federjs. federpy only supports generating visualizations and cannot access it directly. Maybe you can try to use faiss to get it.