Document flattening throws error with deeper nested structures

[jperasmus]

Description

This extension currently flattens the document objects before writing them to Typesense to support older versions of Typesense that didn't support nested data structures. Since v0.24 this is no longer the case and this flattening causes issues.

Here is a stripped-down example of a schema that throws an error when used with this extension (it works fine if you write directly via the API without flattening the data structure):

{
  "name": "collection-name",
  "fields": [
    {
      "name": ".*",
      "type": "auto",
      "facet": false,
      "index": true,
      "sort": false
    },
    {
      "name": "someObject.field1",
      "type": "string",
      "facet": true,
      "index": true,
      "sort": true,
      "optional": false
    },
    {
      "name": "someObject.someObjectArray",
      "type": "object[]",
      "facet": true,
      "index": true,
      "sort": false,
      "nested": true,
      "nested_array": 1
    },
    {
      "name": "someObject.someObjectArray.field2",
      "type": "string[]",
      "facet": true,
      "index": true,
      "sort": false
    }
  ],
  "enable_nested_fields": true
}

I managed to get around the problem by setting "optional: true" on "someObject.someObjectArray.field2", but it is not ideal because I want to make sure that property always exists.

Steps to reproduce

Create a Typesense collection and Firestore collection that resembles the schema given above and trigger either the backfill or onWrite cloud function. It will fail with an error that the schema expects a field named "someObject.someObjectArray.field2" but it doesn't exist on the document.

Expected Behavior

No errors and it indexes the document in Typesense

Actual Behavior

Throws an error

Metadata

Typesense Version:

v0.24

OS: n/a

[jasonbosco]

@jperasmus Thank you for PR #45 that addresses this issue.

I've published the change in v1.0.0-rc.4.

Could you visit the link above and use the installation link under "Installation Instructions" and let me know how it goes?

[jperasmus]

@jasonbosco I've tested this RC version and it is working well. Thanks!