search

mikedilger / chorus

Chorus is a nostr relay
Other
40 stars 7 forks source link

If scraping is not recommended, why have a `created_at+id` index? #18

Closed fiatjaf closed 2 months ago

fiatjaf commented 3 months ago

Isn't the only use of that index to serve queries that don't contain any tags, any authors and any kinds?

(this is just a humble question)

mikedilger commented 3 months ago

You make a good point.

I added the 'ci_index' to improve scraping here:

3ee285b7b7a261d14e2a19cdec04d6f77860f991 "store: Rename indexes; Improve scraping (use since and until on ci_index)"

At that time, specifying a single tag or a single author was also falling into the 'scraping' code path else-block that had no index to use at all. After talking to @staab it became clear that there were some reasonable queries that Coracle was making that were falling into that code path. So I created this ci_index to speed that up.

Later (that same day!) I added a tc_index and an ac_index, which are used when there is at least one tag or author specified. Now that these exist, the ci_index should probably be retired.

mikedilger commented 2 months ago

Actually, if you have a very open filter like just ["REQ", "_", {"kinds": [30818], "limit": 50}] that would get served by the ci index, and serve the 50 events in backwards-time-order. It needs the created_at part of the index to get those particular most recent events.