[BUG] FAISS IVF PQ training throws action_request_validation_exception when nlists parameter is specified

[stephenleo]

What is the bug? Specifying any value for nlists (even default value of 4) during training of FAISS IVF-PQ model throws an error

How can one reproduce the bug?

1. I have ~100K 384d vectors in train-index
2. Try to train the model

   POST /_plugins/_knn/models/my-model/_train
   {
     "training_index": "train-index",
     "training_field": "train-field",
     "dimension": 384,
     "description": "My models description",
     "method": {
         "name":"ivf",
         "engine":"faiss",
         "space_type": "l2",
         "parameters":{
           "nlists": 2048,
           "nprobes": 128,
           "encoder":{
               "name":"pq",
               "parameters":{
                   "code_size": 8,
                   "m": 8
               }
           }
         }
     }
   }

3. Error!

   {
     "error" : {
       "root_cause" : [
         {
           "type" : "action_request_validation_exception",
           "reason" : "Validation Failed: 1: Invalid parameter for method \"ivf\".;"
         }
       ],
       "type" : "action_request_validation_exception",
       "reason" : "Validation Failed: 1: Invalid parameter for method \"ivf\".;"
     },
     "status" : 400
   }

4. Removing the nlists specification from parameters works without error

What is the expected behavior? Model should train with nlists parameter specified

What is your host/environment?

• OS: Docker (https://opensearch.org/samples/docker-compose.yml)
• Version: 1.3.2

Do you have any screenshots? NA

Do you have any additional context? NA

Any help is highly appreciated!

[jmazanec15]

Hi @stephenleo , oops I think this is a documentation bug. The parameter is nlist, not nlists.

Ill go ahead and open an issue to fix it here: https://opensearch.org/docs/latest/search-plugins/knn/knn-index/#ivf-parameters

[jmazanec15]

Cloising in favor of https://github.com/opensearch-project/documentation-website/issues/669. PR has been submitted.