Preferred way to save / pickle trained model

[gosixl]

Hi everyone,

After loading your dataset into the model, how do people think about saving the trained model to be accessed when deployed?

I save it as an .ann file (see code below). However the file gives much lower quality answers when deployed to a server. I've tried hashing the local vs deployed .ann files and I get different hashes.. I don't know what changes during deployment. Is this something folks have experienced?

Code: Training and saving locally

# ... loading dataset
dataset = FCLIPDataset('app', image_source_path='....', image_source_type='URL', catalog=json_data)
fclip = FashionCLIP('laion/CLIP-ViT-B-32-laion2B-s34B-b79K', dataset) 
fclip.nn_index.save('faiannfile.ann') # Save the fclip object

Code: Accessing the file

def fai_search():
    s = request.json['search']   
    n = request.json['n']
    if n is None:
        n = 30
    else:
        n = int(n)

    global fc_data
    if fc_data == None:
        fc_data = AnnoyIndex(512, "dot")
        fc_data.load('faiannfile.ann')
        print(f"File hash: {calculate_file_hash('faiannfile.ann')}")
    else:
        print("Using existing model")

    fc = FashionCLIP('laion/CLIP-ViT-B-32-laion2B-s34B-b79K')
    text_embeddings = fc.encode_text([s], batch_size=8)
    normalised_text_embeddings = text_embeddings / np.linalg.norm(text_embeddings, ord=2, axis=-1, keepdims=True)
    candidates = []
    for v in normalised_text_embeddings:
        result = fc_data.get_nns_by_vector(v, n, search_k=-1, include_distances=True)```

[patrickjohncyh]

Hi gosixl,

The different hashes from the local and deployed file is expected behavior, given that the approx NN index will be generated differently each time. That said, using a locally generated index should not, in theory, change performance when deployed.

Could you provide further information on the performance difference, and your deployment set-up? It would be great if there is a way for us to reproduce the result.

[patrickjohncyh]

reopening

[gosixl]

Thanks Patrick, may you please give me a week or 2 as putting out other fires! A bit of an AI newbie too so bare with me!

[vinid]

closing this in the meantime