jelmerk / hnswlib

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs
Apache License 2.0
260 stars 56 forks source link

how to save the hnsw_index to local disk? and does the cosine distance is right? #19

Closed jiejie1993 closed 4 years ago

jiejie1993 commented 4 years ago

Q1: I have tried many tools to save the "index",but not success, when read the saved "index" from the disk, the "find_neareast"result is "0", can you give the code to save the "index" to disk?

Q2: I test the cosine distance, found that the cosine distance between two same vector is "0", I supose the result should be "1", does it wrong?

jiejie1993 commented 4 years ago

here is my code to save and restore index, but cannot save/restore the index model.

`public void saveIndex(File file) { ByteArrayOutputStream in = new ByteArrayOutputStream(); FileOutputStream outputStream = null; try { outputStream = new FileOutputStream(file); } catch (FileNotFoundException e) { e.printStackTrace(); } try { index.save(in); outputStream.write(in.toByteArray()); } catch (IOException e) { e.printStackTrace(); } }

public HnswIndex<String, float[], TestItem, Float> loadIndex(File file) throws IOException{ FileInputStream fis = new FileInputStream(file); BufferedInputStream bis = new BufferedInputStream( fis ); ByteArrayOutputStream bos = new ByteArrayOutputStream();

    byte[] buf = new byte[1024];
    long len = 0 ;
    long start  = System.currentTimeMillis() ;
    while((len=bis.read(buf, 0, buf.length))!=-1){
        bos.write( buf ) ;
    }

    HnswIndex<String, float[], TestItem, Float> loadedIndex = HnswIndex.load(new ByteArrayInputStream(bos.toByteArray()));
    System.out.println(  System.currentTimeMillis() -start );
    bis.close() ;
    bos.close() ;

    return loadedIndex;
}

`