Closed hiibolt closed 2 months ago
Thank you for reporting the error.
Your observation that both of the fields should be pub
, however, it seems that for correct "shape" of data the following is the correct representation:
pub enum VectorStoreFileObjectChunkingStrategy {
/// This is returned when the chunking strategy is unknown. Typically, this is because the file was indexed before the `chunking_strategy` concept was introduced in the API.
Other,
Static{ static: StaticChunkingStrategy },
}
You're welcome to see that above works and PR is most welcome!
It appears that OpenAI also now requires a name when creating a vector store:
Error: JSONDeserialize(Error("invalid type: null, expected a string", line: 4, column: 14))
(Reproducing code)
// Create a vector store
let vector_store_handle = client
.vector_stores()
.create( CreateVectorStoreRequest {
file_ids: None,
name: None,
expires_after: None,
chunking_strategy: None,
metadata: None
})
.await?;
The fix seems to be changing the following type Option<String>
to String
in the following snippet:
pub struct CreateVectorStoreRequest {
...
#[serde(skip_serializing_if = "Option::is_none")]
pub name: Option<String>,
...
}
I'll write a test covering both and open a pull request shortly, thank you for your fast help!
Thank you for th PR. The change related to vector store name and the error message is not making sense to me, can you elaborate?
Upon further investigation, it appears that there's another bug (because of inconstency in the spec).
You can actually create a vector store by just providing file_ids, in which case name will be null:
2024-06-07T21:37:30.842537Z ERROR async_openai::error: failed deserialization of: {
"id": "vs_kyLc5xI5qptNc3Wd1JHyhmU7",
"object": "vector_store",
"name": null,
"status": "in_progress",
"usage_bytes": 0,
"created_at": 1717796250,
"file_counts": {
"in_progress": 1,
"completed": 0,
"failed": 0,
"cancelled": 0,
"total": 1
},
"metadata": {},
"expires_after": null,
"expires_at": null,
"last_active_at": 1717796250
}
Error: JSONDeserialize(Error("invalid type: null, expected a string", line: 4, column: 14))
That means the actual bug is in VectorStoreObject
where name
needs to be Option<String>
instead of String
.
So you can safely undo the name
field change for CreateVectorStoreRequest
in your PR. and feel free to include the changes for observation above.
Error:
Failing code:
Looks like the issue is here:
I'm new to Rust, but it seems like the
max_chunk_size_tokens
andchunk_overlap_tokens
fields may need to bepub
?