microsoft / SPTAG

A distributed approximate nearest neighborhood search (ANN) library which provides a high quality vector index build, search and distributed online serving toolkits for large scale vector search scenario.
MIT License
4.77k stars 581 forks source link

index.Save function doesn't create the file on disk #390

Closed ChriStingo closed 1 year ago

ChriStingo commented 1 year ago

Describe the bug I'm trying to create an index and, once populated, save it into the disk at a chosen path to load it later in order to perform a search. This doesn't work... the python code terminates without errors, but doesn't save any files to the path passed to the Save() function.

To Reproduce Execute this code with the appropriate modifications.

def create_sptag_index():
    sptag_index = SPTAG.AnnIndex('SPANN', 'Float', get_dataset_columns())
    # add build params
    return sptag_index

def fill_index(sptag_index):
    matrix = []
    # fill the matrix
    m = ''
    for i in range(len(matrix)):
        m += str(i) + '\n'
    sptag_index.BuildWithMetaData(np.asmatrix(matrix), m, len(matrix), False, False)
    sptag_index.AddWithMetaData(np.asmatrix(matrix), m, len(matrix), False, False)

def build_and_save_sptag_index(sptag_index):
    sptag_index.Save(PATH_INDEX) # HERE IS THE GUILTY FUNCTION

def main():
    sptag_index = create_sptag_index()
    fill_index(sptag_index)
    build_and_save_sptag_index(sptag_index)

if __name__ == "__main__":
    main()

Expected behavior Create the file specified as a parameter (in the path specified as a parameter)

Setup: Docker image/container on Ubuntu 22.04 created after executing docker build -t sptag.

ChriStingo commented 1 year ago

I found a way to make Save work. It didn't work because an index of the SPANN type was created during the creation phase. The library did not return any type of error, but replacing with an index of type BKT everything worked.