[Bug]: MilvusException: <MilvusException: (code=1100, message=create index on 104 field is not supported: invalid parameter[expected=supported field][actual=create index on 104 field])>

[takan55]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version:
milvus==2.3.8
milvus-lite==2.4.10
milvus-model==0.2.5
pymilvus==2.4.6
- Deployment mode(standalone or cluster):
milvus lite
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
Mac
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Following the demo https://github.com/milvus-io/bootcamp/blob/master/bootcamp/RAG/sparse_dense_embeddings_tutorial.ipynb or https://milvus.io/docs/hybrid_search_with_milvus.md, this error occurs.

MilvusException: <MilvusException: (code=1100, message=create index on 104 field is not supported: invalid parameter[expected=supported field][actual=create index on 104 field])>

at the code

sparse_index = {"index_type": "SPARSE_INVERTED_INDEX", "metric_type": "IP"}
dense_index = {"index_type": "FLAT", "metric_type": "COSINE"}
col.create_index("sparse_vector", sparse_index)
col.create_index("dense_vector", dense_index)

Expected Behavior

It should work like in demo code.

Steps To Reproduce

Use milvus lite on these demo.

Milvus Log

No response

Anything else?

No response

[yanliang567]

/assign @liliu-z /unassign

[liliu-z]

/assign @junjiejiangjjj

[junjiejiangjjj]

hi @takan55 , may I ask what environment you are running in? I have successfully run these two examples in colab

[HeChengHui]

@junjiejiangjjj

I am also facing the same error. steps: compose up from here, pip install pymilvus.

Env

milvus-lite 2.4.10 pypi_0 pypi pymilvus 2.4.6 pypi_0 pypi

code

index_params = {
    'metric_type':'COSINE',
    'index_type':"GPU_IVF_FLAT",
    'params':{"nlist": 2048}
}

error pymilvus.exceptions.MilvusException: <MilvusException: (code=1100, message=metric type not found or not supported, supported: [L2 IP]: invalid parameter[expected=valid index params][actual=invalid index params])>

[liliu-z]

@junjiejiangjjj

I am also facing the same error. steps: compose up from here, pip install pymilvus.

Env

milvus-lite 2.4.10 pypi_0 pypi pymilvus 2.4.6 pypi_0 pypi

code

index_params = {
    'metric_type':'COSINE',
    'index_type':"GPU_IVF_FLAT",
    'params':{"nlist": 2048}
}

error pymilvus.exceptions.MilvusException: <MilvusException: (code=1100, message=metric type not found or not supported, supported: [L2 IP]: invalid parameter[expected=valid index params][actual=invalid index params])>

From the error msg, you need to specify the metric type. BTW, Milvus-lite doesn't support indexes other than FLAT for now.

[HeChengHui]

@liliu-z

I did specify metric type as 'COSINE' in the index_params i posted. how do i install full milvus as pymilvus installs it?

[junjiejiangjjj]

@liliu-z

I did specify metric type as 'COSINE' in the index_params i posted. how do i install full milvus as pymilvus installs it?

GPU_IVF_FLAT only supports L2 and IP, if you want to use COSINE, you need to use other CPU indexes.

[HeChengHui]

@junjiejiangjjj Noted. is there plans to support cosine soon? i am currently normalising the embs and usng IP metric.

[Presburger]

@HeChengHui hi, This needs to be tracked to see if NVIDIA RAFT has a corresponding support plan. I'll follow up on this soon.

[xiaofan-luan]

@junjiejiangjjj Noted. is there plans to support cosine soon? i am currently normalising the embs and usng IP metric.

One way to use cosine index is to normalize data by yourself.

After normalization, IP is equal to cosine metrics

[chandrakiran428]

Im getting the same error mentioned MilvusException: <MilvusException: (code=1100, message=create index on 104 field is not supported: invalid parameter[expected=supported field][actual=create index on 104 field])>

my pymilvus version is 2.4.7

[xiaofan-luan]

Im getting the same error mentioned MilvusException: <MilvusException: (code=1100, message=create index on 104 field is not supported: invalid parameter[expected=supported field][actual=create index on 104 field])>

my pymilvus version is 2.4.7

What kind of milvus are u using? If shows that you can not create index on this field. So it might be you are trying to create index on an invalid field

[yanliang567]

/assign @chandrakiran428 any updates for this issue?