Unable to import collection item with non-initialized array field

[Khatunzev-Anton]

Description

Hi, We have a problem with import procedure. I have prepared small example. Lets have a collection of following structure:

REQUEST ` POST /collections HTTP/1.1 Host: XXXX:8108 X-TYPESENSE-API-KEY: XXXXXXXXXXXXXXXXXXXXX Content-Type: application/json Content-Length: 340

{ "name": "astra_members_test", "fields": [ { "name": "id", "type": "string", "facet": false }, { "name": "importuris", "type": "string[]", "facet": true }, { "name": "scores_activity_rank", "type": "float", "facet": false } ], "default_sorting_field": "scores_activity_rank" } `

So we have one field of type "string[]". And let us try to upload some data into this collection:

type CollectionItem struct { ID string json:"id" Importuris []string json:"importuris" Scores_activity_rank float64 json:"scores_activity_rank" }

func main() { client := typesense.NewClient( typesense.WithServer("http://XXXX:8108/"), typesense.WithAPIKey("XXXXXXXXXXXXXXXXXXXXX"))

_, err := client.
    Collection("astra_members_test").
    Documents().
    Import(
        []interface{}{
            CollectionItem{
                ID:                   "1",
                Importuris:           []string{"a", "b", "c"},
                Scores_activity_rank: 0.5,
            },
            CollectionItem{
                ID:                   "2",
                Scores_activity_rank: 0.5,
            },
            CollectionItem{
                ID:                   "3",
                Importuris:           []string{},
                Scores_activity_rank: 0.5,
            },
        },
        &api.ImportDocumentsParams{
            Action:    "upsert",
            BatchSize: 500,
        },
    )
if err != nil {
    panic(err)
}
fmt.Println("all done")

}

As you can see, we did not initialize array field for one of our collection items. Now lets check what is in there:

REQUEST

GET /collections/astra_members_test/documents/export HTTP/1.1 Host: XXXX:8108 X-TYPESENSE-API-KEY: XXXXXXXXXXXXXXXXXXXXX

RESPONSE

{"id":"1","importuris":["a","b","c"],"scores_activity_rank":0.5} {"id":"3","importuris":[],"scores_activity_rank":0.5}

So the second item has not been imported. No error returned. If we initialize array field for item#2 it gets imported properly. It seems like a serious limitation and incorrect behavior(according error handling).

Expected Behavior

The item with uninitialized array field gets imported with other items OR proper error returned.

Actual Behavior

The imported collection does not contain the item, no error returned

Metadata

Typsense Version: v0.2.0

OS:

[v-byte-cpu]

Hi @Khatunzev-Anton, can you please provide typesense server version ? Import function returns array with responses for individual documents:

Import(documents []interface{}, params *api.ImportDocumentsParams) ([]*api.ImportDocumentResponse, error)

type ImportDocumentResponse struct {
    Success  bool   `json:"success"`
    Error    string `json:"error"`
    Document string `json:"document"`
}

It reflects current behavior of typesense import API.

[Khatunzev-Anton]

hi @v-byte-cpu, the server version is 0.18.0. BTW i think the problem is in go-client. Initially I tried to prepare the example for you using just SHELL requests. And it all behaved properly. When i tried to import jsonl file containing the item having missing array field - server responded with error and the whole import failed. But with go-client it appears that it is possible to perform "partial" imports)). I will check the ImportDocumentResponse a bit later - it is interesting what is there. Still it would be nice if TS server would allow an import of items with non-initialized array field.