Proposal for improving indexing of tags

[stereosteve]

All the storage adapters loose the association between tag key and value and generally just query by value. There's kinda some todos about this in the code and conversation about it on the sqlite3 PR.

But now I'm like "couldn't we just concatenate the key and value together and index that?" In general a storage adapter will index the tags outside of the actual event.tags field for better performance, so storage adapter can do the same at query time.

So the plan:

• postgres: update stored procedure to index t->>0 || t->>1, update query to query with key || value too. Would also need to do some one time migration to recompute stored values for existing installs.
• sqlite3: add a new tag table that is id, key, value... this one can do the more traditional sql thing similar to how it's done in nostr-rs-relay
• elasticsearch: add a new tags field to IndexedEvent that indexes key || value similar to postgres (since ES indexes are similar to gin indexes anyway... follow same approach). I won't worry about an update migration since I doubt anyone is using it except me.

Anyway I'm happy to take a stab at this if it sounds good. Just wanted to check that it'd be okay to index the concatenated key + value and that I'm not missing something.

[fiatjaf]

Awesome idea.

Yes, if you want to work on it that will be great.

But let me ask first: are you planning on using Relayer for anything?

[stereosteve]

Cool will do.

I'm just experimenting with it for now, but like that it's pretty simple and flexible as compared to the other relays... more of a simple framework than a baked-binary relay. So my interest would be in some custom auth (a-la whitelist example) plus also maybe some custom mirroring to other relays and maybe some traditional REST / opengraph server side rendering stuff eventually.

In reality it's just a side project that will probably fizzle out and die, but it's nice to work on relayer as a way to learn about the nostr.

Why you ask would you advise against using it in "prod" (whatever prod means)?

[stereosteve]

but if you're saying hack on the app not relayer... I hear you. Client is just a CRUD app with two screens atm, but hope to work on it and put it up on the github in a week or two. I'm pointing it at a relayer server to see what's useful or missing.