performant-software / core-data-cloud

Core Data. In the Cloud.
0 stars 1 forks source link

Can't facet by multiple relations with the same model #142

Closed camdendotlol closed 6 months ago

camdendotlol commented 8 months ago

La Supplique has three different works-people relations: Commandement, Titulature, and People Mentioned.

When these relations are indexed, they're all added to a related_people array field. This field stores the type of the relation in a type field on each object in the array. This makes it simple enough to display the desired item in the Hit component by calling some custom logic, e.g. hit.related_people.find(p => p.type === 'Commandement').

However, we can't facet by a specific type of relation. We can facet by each field of the objects in the related_people array, such as related_people.type, related_people.name, etc., but there's no way to facet by Commandement.

jamiefolsom commented 8 months ago

@dleadbetter this is something we need to solve for Sapientia and NBU, because NBU has the same multi-model-inclusion issue.

jamiefolsom commented 8 months ago

Per our discussion, we can create more than one relation of a given model with another given model (person <> person, for parent <> child, sibling <> sibling, etc); we want it to be possible to view each of these relations as a separate facet for search purposes.

For example:

dleadbetter commented 7 months ago

Typesense Issue

Currently there appears to be a bug in Typesense which does not allow defining facets for a schema that also uses auto-type detection. For example, if we have the following field defined in the schema:

{
  name: 'related_people.*_facet',
  type: 'auto',
  facet: true
}

This will result the in field related_people.names_facet not properly being set as a facetable field (I've tried multiple combinations of the above regex with no success).

This poses an issue with user-defined fields, as they will be dynamically added to the schema at index time.

Possible Work-Around

Instead of relying on Typesense to correctly mark the fields in the schema as facet: true, we could instead run a post-processing script that would run after the records have been index to lookup all of the fields in the schema with the _facet prefix and set the "facet" attribute to true. We'll need to keep in mind that the Typesense API does not support modifying fields. Instead we'll need to drop the existing field and add a new one.

dleadbetter commented 7 months ago

@camdendotlol - Thoughts on this as the new Typesense document structure (using a Georgia Coast Atlas place as an example)?

Georgia Coast Atlas

{
  "coordinates": [
    31.4252249,
    -81.2653727
  ],
  "8fb2140d-90da-401d-8f5e-972418738479": "Library",
  "8fb2140d-90da-401d-8f5e-972418738479_facet": "Library",
  "55cefa85-1a9a-43d7-a652-64fb1ec3b851": "https://dvl.ecdsdev.org/items/show/11612",
  "31778cc9-4481-42f0-9978-e8cbf0341ae3": "Environmental Research",
  "31778cc9-4481-42f0-9978-e8cbf0341ae3)facet": "Environmental Research",
  "7a42de29-e8b3-4e8d-8725-1ed157a3f956": "This library was founded in 2022 by Sapelo Island Cultural and Revitalization Society Inc. (SICARS).  The library's location is Sapelo Island's former two-room schoolhouse in the Hog Hammock Community. It is one of the last intact island-based Gullah-Geechee communities in America.",
  "b5525344-4b92-46ec-88c3-a209fafeab64": "11612",
  "d5548c11-a2ba-430d-98ee-8d2c8d2019f7": "Emory ECDS",
  "d5548c11-a2ba-430d-98ee-8d2c8d2019f7_facet": "Emory ECDS",
  "geometry": {
    "coordinates": [
      -81.2653727,
      31.4252249
    ],
    "type": "Point"
  },
  "id": "883",
  "name": "Hog Hammock Public Library (Sapelo Island School)",
  "names": [
    "Hog Hammock Public Library (Sapelo Island School)"
  ],
  "record_id": "4639",
  "e641bbbc-5253-4f06-a156-94b1952e31df": [ // "Media Attachment" relationship
    {
      "name": "hag-hammock-exterior-3.jpg",
      "record_id": "14",
      "uuid": "5f3c2a5e-385e-47d6-a56b-c964d72737e3"
    },
    {
      "name": "hog-hammock-library.jpeg",
      "record_id": "13",
      "uuid": "49123f66-6a4d-47b8-a781-0f31ab63adae"
    }
  ],
  "03590c12-eea4-4f63-b6e8-42e02f614ddc": [ // "Organizations" relationship
    {
      "description": "Cras mi pede, malesuada in, imperdiet et, commodo vulputate, justo. In blandit ultrices enim. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Proin interdum mauris non ligula pellentesque ultrices. Phasellus id sapien in sapien iaculis congue. Vivamus metus arcu, adipiscing molestie, hendrerit at, vulputate vitae, nisl.\n",
      "name": "Bins and Sons",
      "names": [
        "Bins and Sons"
      ],
      "names_facet": [
        "Bins and Sons"
      ],
      "record_id": "4",
      "uuid": "9dbbf99c-1090-4212-ba1d-e6c4155f0ec0"
    },
    {
      "description": "In sagittis dui vel nisl. Duis ac nibh. Fusce lacus purus, aliquet at, feugiat non, pretium quis, lectus. Suspendisse potenti. In eleifend quam a odio. In hac habitasse platea dictumst. Maecenas ut massa quis augue luctus tincidunt. Nulla mollis molestie lorem. Quisque ut erat.\n",
      "name": "Hayes LLC",
      "names": [
        "Hayes LLC"
      ],
      "names_facet": [
        "Hayes LLC"
      ],
      "record_id": "5",
      "uuid": "38ca082b-96fd-4ef2-b107-f827f6c1005a"
    },
    {
      "description": "Aliquam quis turpis eget elit sodales scelerisque. Mauris sit amet eros. Suspendisse accumsan tortor quis turpis. Sed ante. Vivamus tortor. Duis mattis egestas metus.\n",
      "name": "Little Group",
      "names": [
        "Little Group"
      ],
      "names_facet": [
        "Little Group"
      ],
      "record_id": "6",
      "uuid": "c7d26821-2cf3-41e6-8fea-a7d298e7e62e"
    }
  ],
  "1b43ada9-652e-4505-b9d7-f1cc34b521f2": [ // "People" relationship
    {
      "biography": "Suspendisse potenti. In eleifend quam a odio. In hac habitasse platea dictumst. Maecenas ut massa quis augue luctus tincidunt. Nulla mollis molestie lorem. Quisque ut erat. Curabitur gravida nisi at nibh. In hac habitasse platea dictumst. Aliquam augue quam, sollicitudin vitae, consectetuer eget, rutrum at, lorem.\n",
      "name": "Francis D'Hooge",
      "names": [
        "Francis D'Hooge"
      ],
      "names_facet": [
        "Francis D'Hooge"
      ],
      "record_id": "16",
      "uuid": "123d0955-7125-402a-b404-3093284b421c"
    },
    {
      "biography": "In congue. Etiam justo. Etiam pretium iaculis justo. In hac habitasse platea dictumst. Etiam faucibus cursus urna. Ut tellus. Nulla ut erat id mauris vulputate elementum. Nullam varius. Nulla facilisi.\n",
      "name": "Augustine Gunbie",
      "names": [
        "Augustine Gunbie"
      ],
      "names_facet": [
        "Augustine Gunbie"
      ],
      "record_id": "14",
      "uuid": "a36f53ef-023f-46b3-80ee-773b82b49e96"
    },
    {
      "biography": "Mauris enim leo, rhoncus sed, vestibulum sit amet, cursus id, turpis. Integer aliquet, massa id lobortis convallis, tortor risus dapibus augue, vel accumsan tellus nisi eu orci. Mauris lacinia sapien quis libero. Nullam sit amet turpis elementum ligula vehicula consequat. Morbi a ipsum. Integer a nibh. In quis justo. Maecenas rhoncus aliquam lacus. Morbi quis tortor id nulla ultrices aliquet.",
      "name": "Olivette Ivory",
      "names": [
        "Olivette Ivory"
      ],
      "names_facet": [
        "Olivette Ivory"
      ],
      "record_id": "15",
      "uuid": "405c5d21-c693-4212-83f2-3708570fa93e"
    }
  ],
  "13c179ac-99c2-4b84-a48d-c8540c1119b2": [ // "Related Places" relationship
    {
      "8fb2140d-90da-401d-8f5e-972418738479": "Island",
      "8fb2140d-90da-401d-8f5e-972418738479_facet": "Island",
      "55cefa85-1a9a-43d7-a652-64fb1ec3b851": "https://dvl.ecdsdev.org/items/show/11451",
      "31778cc9-4481-42f0-9978-e8cbf0341ae3": "Environmental Research",
      "31778cc9-4481-42f0-9978-e8cbf0341ae3_facet": "Environmental Research",
      "7a42de29-e8b3-4e8d-8725-1ed157a3f956": "Sapelo Island is home to the Gullah/Geechee people, many of whom reside at Hog Hammock on the southern half of the island. The rest of Sapelo is state-owned and managed. Its ecosystem and long history of occupation make Sapelo a site of great ecological, archaeological, cultural, and historical significance.",
      "b5525344-4b92-46ec-88c3-a209fafeab64": "11451",
      "d5548c11-a2ba-430d-98ee-8d2c8d2019f7": "Emory ECDS",
      "d5548c11-a2ba-430d-98ee-8d2c8d2019f7_facet": "Emory ECDS",
      "geometry": {
        "coordinates": [
          -81.2417611,
          31.4774455
        ],
        "type": "Point"
      },
      "name": "Sapelo Island",
      "names": [
        "Sapelo Island"
      ],
      "names_facet": [
        "Sapelo Island"
      ],
      "record_id": "4478",
      "uuid": "ad02a467-5852-4b64-a778-5fa57e1c354c"
    }
  ],
  "9f53e922-a68e-44c7-b7f9-f246050cda54": [ // "TopoName" relationship
    {
      "name": "Doboy Sound",
      "name_facet": "Doboy Sound",
      "record_id": "1",
      "uuid": "ec79302a-0590-4ba5-b9bd-fbf86c4bb3b3"
    }
  ],
  "uuid": "3aaf97a4-7052-4e2c-9056-4f4146ef0c87"
}
camdendotlol commented 7 months ago

That looks good to me. I assume it would be possible to facet on nested values like 9f53e922-a68e-44c7-b7f9-f246050cda54.name?

dleadbetter commented 7 months ago

That looks good to me. I assume it would be possible to facet on nested values like 9f53e922-a68e-44c7-b7f9-f246050cda54.name?

Yes, ideally any field with _facet in the name (root level or nested) will be facetable. For models with multiple names, we'll likely make the names array facetable instead of just the primary name.

dleadbetter commented 6 months ago

Nothing to test on the Core Data CMS. Testing done by websites consuming the API. Closing.