Removes filename from file upload schema: specifying a preferred filename was a low-value feature that introduced complexity. Intention was to specify a preferred filename in the UI which would appear when the file was added to an assistant instead of the file name as it exists on the computer at time of upload but the functionality got overlooked in the frontend implementation, so I decided to remove it from the backend since it wasn't being used.
Modifies the vector schema to move everything that isn't the namespace and embedded content to metadata, allowing for compatibility when external tools and vector databases are used with the platform.
Allows score threshold to be set (see below)
Schema Changes
For JSON and CSV file Upload, there is a new parser_config in the IngestRequestPayload schema that contains a structured_data_content_field. When the file contents are looped through, the contents of structured_data_content_field is sent through the ingestion pipeline and all other fields for the respective object are automatically added as chunk metadata
The EncoderConfig now include score_threshold, allowing for the score threshold of retrieved results to be modified. This is reflected as an optional field in the encoder config portion of the ingest and query APIs.
34 Adds support for JSON file upload
34 Adds support for CSV file upload
filenamefrom file upload schema: specifying a preferred filename was a low-value feature that introduced complexity. Intention was to specify a preferred filename in the UI which would appear when the file was added to an assistant instead of the file name as it exists on the computer at time of upload but the functionality got overlooked in the frontend implementation, so I decided to remove it from the backend since it wasn't being used.Schema Changes
parser_configin the IngestRequestPayload schema that contains astructured_data_content_field. When the file contents are looped through, the contents of structured_data_content_field is sent through the ingestion pipeline and all other fields for the respective object are automatically added as chunk metadata