toluaina / pgsync

Postgres to Elasticsearch/OpenSearch sync
https://pgsync.com
MIT License
1.1k stars 172 forks source link

Support concat for children node fields #461

Open abdullah2993 opened 1 year ago

abdullah2993 commented 1 year ago

It would be really nice if concat transform could support the ability to concat child columns in parent e.g.

 {
    "database": "postgres",
    "index": "products",
    "nodes": {
      "table": "products",
      "children": [
        {
          "table": "vendors",
          "label": "vendor",
          "columns": [
            "name"
          ],
          "relationship": {
            "variant": "object",
            "type": "one_to_one"
          }
        },
        {
          "table": "line_items",
          "label": "items",
          "columns": [
            "description"
          ],
          "relationship": {
            "variant": "object",
            "type": "one_to_many",
            "foreign_key": {
              "child": [
                "product_id"
              ],
              "parent": [
                "id"
              ]
            }
          }
        }
      ],
      "columns": [
        "number",
        "purchase_order",
        "sku",
        "quantity",
        "stock_date",
        "amount"
      ],
      "transform": {
        "concat": {
          "columns": [
            "number",
            "purchase_order",
            "billing_address",
            "email_addresses",
            "estimate_date",
            "amount",
            "items.description", <------ support this
            "vendor.name" <------ support this
          ],
          "destination": "global_search_field",
          "delimiter": " "
        }
      }
    }
  }

support concating items.description and vendor.name in the given sample

toluaina commented 11 months ago

I get what you mean. That could get complicated in generalised cases though. e.g if the parent had multiple children then you need to specify in index to concat.

abdullah2993 commented 11 months ago

I'm not sure if I understand the question.

carlos-pereira-costoya commented 5 months ago

@toluaina It would be very interesting in cases of having a one_by_one relationship type to avoid and filter situations with multiple results. Is this issue still open, or is this other implementation for pgsync in some version snapshot? Is there a repository of plugins where I can find something similar? Thank you very much.