moonstream-to / api

Building blocks for your blockchain economy
https://moonstream.to
Apache License 2.0
135 stars 49 forks source link

Json query language for event Filtering. #534

Open Andrei-Dolgolev opened 2 years ago

Andrei-Dolgolev commented 2 years ago
select
    label_data -> 'args' ->> 'to' as purchaser,
    label_data -> 'args' ->> 'tokenId' as tokenId,
    to_timestamp(block_timestamp) as time_of_sale
from
    polygon_labels
where
    label = 'moonworm-alpha'
    and address =  "<address>"
    and label_data ->> 'type' = 'event'
    and label_data ->> 'name' = 'Transfer'
    and label_data -> 'args' ->> 'from' = '0x0000000000000000000000000000000000000000'
    and (label_data -> 'args' ->> 'tokenId') :: int in (
        11000,
        12000,
        13000,
        14000,
        15000,
        16000,
        17000,
        18000,
        19000,
        20000
    )
order by
    tokenId;

Expected Json.

{
  "output_statment": [
    { "block_timestamp": { "type": "timestamp" } },
    { "args": { "name": "to" } },
    { "args": { "name": "from" } }
  ],
  "filters": [
    {
      "address": "<address>",
      "label_filters": [
        {
          "name": "Transfer",
          "type": "metod",
          "args_combinator": "AND",
          "args": [
            {
              "name": "from",
              "value": "0x0000000000000000000000000000000000000000",
              "operation": "="
            },
            {
              "name": "tokenId",
              "type": "int",
              "value": [
                11000, 12000, 13000, 14000, 15000, 16000, 17000, 18000, 19000,
                20000
              ],
              "operation": "in"
            }
          ]
        }
      ]
    }
  ],
  "group_condition": [],
  "order_by": ["tokenId"]
}
peersky commented 2 years ago

@Andrei-Dolgolev I think is must to add blockchain selector as well. And I assume for some contracts it might be useful to actually query for polygon AND/OR ethereum simultaneously

Andrei-Dolgolev commented 2 years ago

I think is must to add blockchain selector as well. And I assume for some contracts it might be useful to actually query for polygon AND/OR ethereum simultaneously

@peersky Yes good idea, but not sure if i ready do it in first iteration for now it can apply to different blockchains via sub: query perameter. But later we need support 2 different query to different blockchain it's probably help with watching any bridge operation. But its meen json query must support same logic as we do with query parametrs rifht now or even simple just fully replace it.

Andrei-Dolgolev commented 2 years ago

Extended version:

{
    "output_statment": [
      { "block_timestamp": { "type": "timestamp" } },
      { "args": { "name": "to" } },
      { "args": { "name": "from" } }
    ],
    "filter_block":{
        "condition": {
            "AND":{
                "address": "<address>",
                "name": "Transfer",
                "type": "metod",
                "condition": {
                    "OR": {
                        "args": [
                            {
                                "name": "from",
                                "value": "0x0000000000000000000000000000000000000000",
                                "operation": "="
                            },
                            {
                                "name": "tokenId",
                                "type": "int",
                                "value": [
                                11000, 12000, 13000, 14000, 15000, 16000, 17000, 18000, 19000,
                                20000
                                ],
                                "operation": "in"
                            }
                        ]
                    }
                }
            }
        }
    },
    "group_condition": [],
    "order_by": ["tokenId"]
}
peersky commented 2 years ago

cant it be just part of args[] with

name: OR
operation: boolean_algebra

And if we args can be recursive than we could have smth like

[
  {
    "name": "from",
    "value": "0x0000000000000000000000000000000000000000",
    "operation": "="
  },
  {
    "name": "OR",
    "operation": "boolean_algebra"
  },
  {
    "name": "from",
    "value": "0x0000000000000000000000000000000000000001",
    "operation": "="
  },
  {
    "name": "AND",
    "operation": "boolean_algebra"
  },
  {
    "name": "nested_args",
    "operation": [
      {
        "name": "to",
        "value": "0x000000000000000000000000000000000000d3ad",
        "operation": "="
      },
      {
        "name": "OR",
        "operation": "boolean_algebra"
      },
      {
        "name": "to",
        "value": "vitalik.eth",
        "operation": "="
      }
    ]
  }
]
zomglings commented 2 years ago

My proposal:

{
  "outputs": [
    {
      "selector": "block_timestamp",
      "as_type": "timestamp",
      "display_name": "time_of_sale"
    },
    {
      "selector": "data.args.to",
      "display_name": "purchaser"
    },
    {
      "selector": "data.args.tokenId",
      "display_name": "tokenId"
    }
  ],
  "order": {
    "display_names": ["tokenId"],
    "sort": ["asc"]
  },
  "filters": {
    "clause_type": "and",
    "subclauses": [
      {
        "clause_type": "or",
        "subclauses": [
          {
            "clause_type": "simple",
            "data.args.from": {
              "comparator": "=",
              "value": "0x0000000000000000000000000000000000000000"
            }
          },
          {
            "clause_type": "simple",
            "data.args.tokenId": {
              "cast": "int",
              "comparator": "in",
              "value": [
                11000, 12000, 13000, 14000, 15000, 16000, 17000, 18000, 19000,
                20000
              ]
            }
          }
        ]
      }
    ]
  }
}
zomglings commented 2 years ago

Alternative - use MongoDB aggregation pipelines syntax: https://docs.mongodb.com/manual/core/aggregation-pipeline/

zomglings commented 2 years ago

First draft implementation: @Andrei-Dolgolev and I agreed to start with MongoDB syntax.

This plays well with the react query builder. For example, this example generates this query:

{"$and":[{"firstName":{"$regex":"^Stev"}},{"lastName":{"$in":["Vai","Vaughan"]}},{"age":{"$gt":"28"}},{"$or":[{"isMusician":{"$eq":true}},{"instrument":{"$eq":"Guitar"}}]}]}
zomglings commented 2 years ago

MongoDB style syntax for our query:

[
  {
    "$match": {
      "data.args.from": "0x0000000000000000000000000000000000000000",
      "data.type": "event",
      "data.name": "Transfer",
      "data.address": "<contract address>",
      "data.args.tokenId": {
        "$cast": "int",
        "$in": [
          11000, 12000, 13000, 14000, 15000, 16000, 17000, 18000, 19000, 20000
        ]
      }
    }
  },
  { "$sort": "data.args.tokenId" },
  {
    "$map": {
      "data.args.to": "purchaser",
      "data.args.tokenId": "tokenId",
      "block_timestamp": "time_of_sale"
    }
  }
]
zomglings commented 2 years ago

Our MongoDB style syntax introduced two extra keys: "$cast" and "$map". Need to think about how to handle with react query builder.

Andrei-Dolgolev commented 2 years ago

Extend are little logic of map because it require more be the same keys

[
  {
    "$match": {
      "data.args.from": "0x0000000000000000000000000000000000000000",
      "data.type": "event",
      "data.name": "Transfer",
      "data.address": "<contract address>",
      "data.args.tokenId": {
        "$cast": "int",
        "$in": [
          11000, 12000, 13000, 14000, 15000, 16000, 17000, 18000, 19000, 20000
        ]
      }
    }
  },
  { 
    "$sort": { "data.args.tokenId": "desc" }
  },
  {
    "$map": [
      {
        "data.args.to": "purchaser"
      },
      {
        "data.args.tokenId": "tokenId"
      },
      {
        "block_timestamp": "time_of_sale"
      }
    ]
  }
]