21centurymotorcompany / bitd

DEPRECATED. Replaced with Planaria https://planaria.network
82 stars 38 forks source link

reversed 'tx.h' is needed for $lookup #3

Open pmitchev opened 5 years ago

pmitchev commented 5 years ago

Memo is using tx.h in reverse form. Something like 'tx.rh' is needed to allow lookups. Unless I am missing a way to do it without additional field of cоurse

unwriter commented 5 years ago

Good point, I don't think we should index it directly to bitdb but could solve this on bitquery level. Maybe could add an additional "preprocessing" (p.f) step to bitquery, the same way it's doing post processing with jq (r.f).

for example,

{
  "v": 3,
  "p": {
    "f": [some jq transform function that reverses the hex]
  },
  "q": {
    "find": {
      "tx.h": "6ef29bf016605e177b6d2c28b104496f0cf1fe3903890848c5856e764fa3bdd6"
    }
  },
  "r": {
    "f": [post processing]
  }
}

Or there can be other approaches. Will think about this.

BTW could you share why it would be beneficial to have bitdb support this query natively (instead of just coding the reverse logic on the app level, outside of the query)? I have my own ideas but just to make sure I'm not missing anything.

pmitchev commented 5 years ago

My issue is for $lookup specifically. But as you said, it probably will perform very bad and $lookup is not the correct way to go. Example of what I want to do:
Order memo posts containing "cats" by likes.

unwriter commented 5 years ago

Aha I see, then this is something that may need some more thought since the processing needs to come during the aggregation, which would mean somehow monkey patching the mongodb query engine if done naively.

But maybe there are other ways of doing this, I'll think about it.