We need a way to request direct replies to a note

[arthurfranca]

EDIT: Hello client authors, please skip to this comment to read the tl;dr of the last version of this proposal.

Currently, if we use the filter { kinds: [1], #e: ['note id'] }, it won't bring only the direct replies to the note, making it hard (impossible?) to lazily (e.g. infinite scroll) build a thread, specially the ones with lots of engagement.

This is because of:

• NIP-08 (Handling Mentions) supporting clients can add extra e tags mentioned in content
• NIP-10 supporting clients can add many e tags (1 root, maybe 1 reply and maybe one or more mentions)

There is no way to fetch just the direct replies (that would be the NIP-10 with the root marker OR the NIP-10 with the reply marker; it depends).

Currently, nostr expects clients to fetch all thread at once (which will probably get limited by relays when the thread is big). It gets really difficult when you use limit on the request filter as it will sort by created_at desc, so it may retrieve latest non-direct replies or mentions. E.g.: { kinds: [1], #e: ['root note id'], limit: 10 } may bring 2 mentions and 8 replies not directed to the root note

So i think we need to add a tag denoting direct reply. Maybe an R tag (just one max per event).

What do you think of adding it so that a client can use the following filter? { kinds: [1], #R: ['note id or long form content address'], limit: 10 }

And this should supersede NIP-10 reply marker.

[staab]

I like this solution a lot. Replies are the one universal concept in social media applications, event tags are far too broad to support them efficiently.

[barkyq]

I like this too. In general, would be nice to have efficient ways to build reply trees.

Use-case 1: Start at a leaf note and move to the root note. Use the IDs field in the query filter, taking the ID from the R tag.

Use-case 2: Start at root note and fetch the child notes (building the reply tree from top down). With this proposal, can use R query tag. Anticipated improvement in bandwidth + specificity of the query. Still will probably return a fair amount spam if spammers include the R tag linking to popular messages. Limiting only one R tag per note could mitigate that a bit though.

[fiatjaf]

In principle this is fine, but this is introducing a huge breaking change that also possibly makes it more inefficient to download a full thread and get live updates from it.

And the goal is just to solve the problem of "thread too big"? What is a thread too big? A thousand replies? Clients are downloading much more than a thousand events routinely on feeds today, for reactions, contact lists and whatnot. How many threads have a thousand replies? Where are these big threads? Are these replies all in the same relay (in the future they probably would not be)?

If a thread has hundreds of replies then I think there are other problems that must be solved before you introduce a way to do these fine-grained queries, because no one is going to read these huge threads which are likely to be full of spam and valueless comments, so clients must already be mindful of querying for events in a thread only in relays that provide good signal (and other techniques).

[barkyq]

I guess the size is relative to the relays max # of events returned per query.

If everyone in the thread agrees to tag the "root", then any query which includes the root tag will return every note in the thread. I suppose one could paginate using since and until.

Theoretically, I think that it would be nice to have some way of asking for the "direct children" of a note (query the R tag). I also think it is nice to be able to ask for all notes which refer to a given note (query the e tag)... These two ways can hopefully coexist. Thus I disagree that it would make it more inefficient to download a full thread, since the proposal would not replace the current way of doing things.

I think that, if a note N is on Relay X, the client should ideally be able to recreate the surrounding "thread reply tree" near N without having to go to other relays. At least, I think this should be a worthwhile goal. I agree that if Relay X is a super spammy relay, then it may be difficult to achieve this, even with @arthurfranca's proposal.

---

I suppose it would be inelegant to include the same data in an R tag and in an e tag. But duplicating the data would be the least breaking change. There could be something like R tag is an integer pointer to the e tags index, and the relay should use the corresponding e tag data when inserting the R tag into an index. Since this is not "generic tag query" behavior, perhaps one would want to switch away from a single letter. Maybe something like dd (for direct descendant). For example:

{
  ...,
  "tags":[["e","aaa..."],["p","ccc..."],["e","bbb..."],["dd","2"]],
  ...
}

This note would be a direct descendant of bbb.... The filter:

{
  "#dd": ["bbb..."],
}

would match the above note.

For reducing amount of logic, the dd tag MUST come after the parent e tag in the list of tags. The dd tag contents MUST be a base 10 integer encoded as a string without leading 0s. The integer MUST be the 0-indexed position of an e tag in the tag array. Any dd tag after the first dd tag SHOULD be ignored by clients and relays implementing this.

Seems like something like this could slowly replace reply markers, without breaking anything too badly.

[barkyq]

It is late here. and perhaps my idea is ridiculous. will leave it up for posterity.

EDIT (added later):

1. I do like that the extra data would be very lightweight for relays/clients which do not implement this...
2. Seems a bit overkill since websocket compression would already compress repeated information quite well.
3. Basically just moving the reply marker to its own tag, modulo the case where there is only the root marker. So seems a bit silly to add a new tag.
4. (tongue-in-cheek) If we go with dd, then it would natural to call this NIP-221 since 0xdd=221.

[aaafrancaaa]

OP here, not main acc @fiatjaf main goals are: 1) reduce bandwidth usage to lower relay hosting costs 2) increase page load speed 3) reduce cumulative layout shift (notes moving up an down while replies come at any order from different branches. All clients i've tested are like that)

2 and 3 are good to ux and ranking better at search engine results.

It is common for a user to read just some comments of a thread and go back to the feed. Very inneficient to request all thread for every user at every client reading a thread.

There could be something like R tag is an integer pointer to the e tags index, and the relay should use the corresponding e tag data when inserting the R tag into an index

@barkyq as you describe it, it seems the relay would have to store it either way so to index the r tag if i got it right. And the two char tag wouldn't get indexed by relays.

[barkyq]

Relays are able to index tags with more than 1 character, it's just not part of the generic tag queries NIP. My thought was, if the desired query behavior does not follow the generic tag query format, it should not use 1 character.

[aaafrancaaa]

Ok got it barkyq. I don't know what the best approach is but i think we do need this feature.

[barkyq]

Well I suppose there are (at least) three options.

Option 1: [["e","aaa..."], ["e","bbb..."], ["R","bbb..."]] Option 2: [["e","aaa..."], ["e","bbb..."], ["dd","1"]] Option 3: [["e","aaa..."], ["e","bbb..."], ["R","1"]]

Option 1 is nice because it leverages existing generic query tag behavior. In this sense, relays do not need to change anything to support this. A bit inelegant because of repeated info. Option 2 is nice because it has a smaller event size. It is opt-in by the relay since it does not use generic query tag format. Option 3 is bad since relays which support generic tag query but not this NIP will index these events as having useless R tag content.

The bandwidth downside to Option 1 is mitigated if compression is used.

[fiatjaf]

It is common for a user to read just some comments of a thread and go back to the feed.

They can already fetch the parent note.

[fiatjaf]

1) reduce bandwidth usage to lower relay hosting costs 2) increase page load speed 3) reduce cumulative layout shift (notes moving up an down while replies come at any order from different branches. All clients i've tested are like that)

1) How? 2) This is not obvious at all. Doing a bunch of queries is probably much slower than a single query -- but of course it depends on many other implementation details and thread size. 3) Web clients are horrible and there is ultimately no way to fix this. Yet it is very possible to make them better and remove these issues without breaking the current standard. Also I am pretty sure that just changing the standard wouldn't automatically make web clients better, that would still require the same amount of work to make the UIs not suck.

[aaafrancaaa]

@fiatjaf for instance, if we consider we can start as low as loading 5 direct children of the main note (depends on note bubble height), here it goes:

1. { kinds: [1], #R: ['note id or long form content address'], limit: 5 } (if multiple relays, req the same for each one, merge and show at UI just the 5 latest replies)
2. Optionally one would want to show a "continue thread" link below each children that has its own children. So for each 5 items we do { kinds: [1], #R: ['note id'], limit: 1 }, limiting to 1 item just to know if there is more or not.
3. User scrolls to bottom
4. Repeat 1 (add until: xxxx), 2 and 3

The inner branches use same algorithm, but triggered by user click instead of scrolling.

It is lighter bandwidth-wise and potentially faster if we consider main content comes from step 1.

[barkyq]

Right now most threads are pretty small so its not a big deal. I do think that this "feature" is a fairly fundamental one though. Clients should be able to say where they want their event to be positioned in the reply tree, in a way that relays can understand, and in a way which other clients can query.

I don't see why there is an emphasis on "breaking the current standard." Seems like this would be an opt-in feature which would not break the old way of doing things. Sure, if a client wanted to implement this, it would require a few new lines of code, but probably they already have that logic with deciding where to add the reply marker to.

Also, sidebar, there is not really a well-adopted "current standard." Half the people use reply and root and the other half just add a bunch of unmarked e tags.

There is certainly some discussion to be had whether this actually achieves anything beyond what can already be achieved using since, until, limit, and #e. Perhaps not. I think the lazy-loading use case described by @aaafrancaaa is nice.

In general, I think there is merit to having collections of events organized in a tree (e.g., posts in a github issues style thing, where each repo is an event, each issue event sets its parent to the repo event, each post sets its parent to the issue event).

[arthurfranca]

In general, I think there is merit to having collections of events organized in a tree (e.g., posts in a github issues style thing, where each repo is an event, each issue event sets its parent to the repo event, each post sets its parent to the issue event).

Yes @barkyq and tag name should be P for Parent event or something like that instead of R. R for direct Reply was kind of an inverted name, cause i was naming similar to NIP-10 reply marker. A tag to indicate the parent note/long form content(/any event) of which the current note is direct descendant.

The NIP-10 is marked as a draft. I can try to edit it, keeping the current way of doing things as it may be preferred by clients, just adding the P tag.

But it seems @fiatjaf isn't convinced.

[fiatjaf]

I like the idea, if it was like that since the beginning it would have probably been great. I am not convinced it's worth changing. If other client developers like it enthusiastically as @staab does, including Damus, we can for sure modify NIP-10 to include this yet new way of replying.

Although r for "root" and p for "direct parent" makes more sense to me -- so exactly the opposite of what you suggested.

[mikedilger]

Also, sidebar, there is not really a well-adopted "current standard." Half the people use reply and root and the other half just add a bunch of unmarked e tags.

Yes, and this suggestion adds a third way. Adding new ways doesn't eliminate the old ways. https://xkcd.com/927/

If everyone were doing marked 'e' tags as recommended in NIP-10, then this suggestion could be implemented entirely by relays, right? And we wouldn't need a new tag, except we would need a new way to specify in the filter which events we were seeking.

[barkyq]

If everyone were doing marked 'e' tags as recommended in NIP-10, then this suggestion could be implemented entirely by relays, right? And we wouldn't need a new tag, except we would need a new way to specify in the filter which events we were seeking.

Yes, except that an e tag cannot be marked reply and root simultaneously.

Although I guess this could be handled (in a bit of a complicated way) by checking if there is only "root" and then assuming that is also supposed to be the "reply".

[barkyq]

I am partial to the addition of tags like ["dd", "3"], where "3" points to the index 3 entry in the tag array.

But I am certainly biased since it was my own idea..

One advantage is that multiple of these pointer tags could point to the same e tag. One disadvantage is new code for both relays and clients (and the XKCD comic's observation). Also having trouble thinking if this type of pointer tag would have any other use cases.

[arthurfranca]

Yes, and this suggestion adds a third way. Adding new ways doesn't eliminate the old ways. https://xkcd.com/927/

At least NIP-10 is a draft, maybe because we are at a time clients are trying things and seeing what works best or maybe we are just lucky lol cause as a draft we theoretically have the possibility to together decide how new and up-to-date social clients should work to interoperate and deprecate the old ways.

If everyone were doing marked 'e' tags as recommended in NIP-10, then this suggestion could be implemented entirely by relays, right? ...

I don't think adding relay logic and complexity, like the "dd" tag @barkyq suggests, just for an use case is better than adding a tag. But of course it is open for discussion.

The way of building all thread at once using the root marker works well (with caveats already listed) when the client is viewing the thread from the beginning. But when e.g. a twitter-like client is showing a view starting from a reply X deep in the tree it doesn't work, cause X isn't marked as root (but yeah, client could still load all thread just to show the slice). Yet, a tag/marker for root is good even when lazy loading, cause it enables showing something like this:

root
... "load more" ...
X
direct replies to X (plus "continue thread" for each one that has children)
... "load more" or infinite scroll ...

We have to think of 3 things: what is best for relays (bandwidth, disk space, simplicity), clients (lazy loading possibility? loading everything at once if it wishes? both or just one way?) and users (ux, notifications – NPT-10 lowercase p tag, discoverability by seo). Just client point of view isn't enough.

I don't see disqus, discourse, fb, instagram, twitter, telegram, whatsapp, reddit (all?) loading all thead at once. But if we still want to support the 2 ways of fetching a thread with minimum change, we could kill reply marker and add the P tag. The only "problem" is when P and e-root (or R tag as @fiatjaf said; uppercase cause r is already taken meaning reference?) have the same value we can't keep just the e-root/R as before because of lazy loading way and can't keep just the P tag because of the load-all-thread-at-once way. Not that bad the two tags having same value in this case (when at first level of replies) i guess.

If we use the R tag for root, we could kill markers, keeping e just for mentions (NIP-08 inline or reposts). R tag would allow long-form content address as value beside event id. Also it would make the load-all-thread-at-once way better by not fetching events that just mention the root when in fact one wants events that have a specific root (filter by R instead of the broader e – same that happens when using P instead of e).

Speaking of markers, now there is #293 pr for p markers which would reintroduce markers. But that should be discussed there.

[arthurfranca]

tl;dr

I've come up with an even better and simple way of connecting events as replies, needed for more advanced queries that would fight spam as a side effect.

---

First, an introduction to explain why it is needed. Yesterday i was paying attention to how hamstr.to ("Hamstr is a twitter-style Nostr web client") load replies compared to twitter and noticed it differs in an important way among other things.

Twitter tries to show the user the most interesting threads (MIT) on a post page first. I think MITs are the conversations/mini-threads that include OP replies or user (the one who is viewing the page) follows' replies – a series of linked replies (on twitter, these are linked by vertical lines between avatars).

MIT's also appear on Instagram (they highlight responses from OP) and on Youtube (they highlight replies liked by OP). MITs are always on top of other replies.

MITs have a great side effect of pushing down least intereresting conversations, including the damned spam, cause below MITs, considering that limit desc sort things on nostr, will be the latest direct descendant replies, which may include spam the relay wasn't able to block. (I'm considering the lazy loading way of loading threads instead of all at once that could sort however one wants).

If we consider MITs are desired, we need a way to fetch them without loading all replies at once, specially when considering spam could make threads huge and all other caveats already listed before. Loading all thread at once is just not best practice.

On twitter, when user clicks on a reply X, a page loads with MITs considering X as the OP of the branch. One example of a 2 levels MIC would be Y replying to X then X replying to Y. This is an important example that shows that sometimes we want MITs that don't consider the root as OP

Current way and my previous proposal aren't enough to request MITs. Recently, I was finally able to come up with a solution that would allow queries such as: show me X-OP direct replies to X OR X-OP/user friends replies to X's direct descendants. Also, show me X's direct descendant replies that were liked by X-OP. Examples later below.

---

New tags:

The branch b tags are responsible for telling what branch of the tree a reply is part of. Each b tag value is the id or address (if replaceable event) of the parent event, grandparent and so on up until reaching the root one. For instance, a reply event D should copy the replied to event C's b tags and then append C event id or address. A root event doesn't have b tags.

The level l tag is responsible for telling at what level in the tree a reply is. A reply event b tag count should be used as l value. A root event l tag has value '0' to allow requesting just root events.

An event can not be a reply to more than one event. (Or else it would mess with the l tag)

A relay may limit events to a certain max level (5 maybe?) <- Clients just need to not append a b tag if already at level 5.

Reaction events (NIP-25) must copy b and l tags from the reacted to event.

Example of a reply to 'ghi...' event:

  {
    ...,
    tags: [
      ['b', 'abc...'] // root event id or address
      ['b', 'def...']
      ['b', 'ghi...'] // replied to event id or address
      ['l', '3']
    ],
  }

Example load-all-thread-at-once filter

  {
    "kinds": [1],
    "#b": ['first X\'s b tag occurence value'], // root event id or address
  }

Example parent filter

  {
    "ids": ['last X\'s b tag occurence value'],
    "kinds": [1],
    "limit": 1
  }

Example direct descendant filter

  {
    "kinds": [1],
    "#b": ['X id or address']
    "#l": ['3'], // considering X is on level 2
    "limit": 5
  }

Example MIT filter

  {
    "authors": [X pubkey, user contact 1, user contact 2...],
    "kinds": [1],
    "#b": ['X id or address']
    "#l": ['3', '4'], // considering X is on level 2
    "limit": 5
  }

Example MIT filter by X-OP reaction

  {
    "authors": [X pubkey],
    "kinds": [7],
    "#b": ['X id or address']
    "#l": ['3'], // considering X is on level 2
    "limit": 5
  }

[arthurfranca]

Please people mention client authors you know so that they see this issue. @jb55 <- Damus (only know this one)

[staab]

I like this in the abstract, the level tags are pretty interesting. This is pretty similar to how e tags were originally specified (which ended up being unreliable because order wasn't specified?). At this point I'm with @fiatjaf because of the compatibility issue. Because a client can't rely on other clients implementing a new tag scheme, they can't rely on the tag scheme. Asking for l=3 will drop everything published by non-conforming clients. Maybe it's not time for the protocol to ossify yet, but that's the feeling I'm starting to get.

An alternative solution to this problem could be to solve it on the relays' side with a new filter, for example ["REQ", "23974", [{"marks": [["<event-id>", "reply"]]}]]. I personally tend to think marks were a mistake (in the vein of d tags, although those are better specified), but they are pretty well supported, and would solve the immediate children problem.

[arthurfranca]

@staab thank you for your feedback as a client author.

~Looking at what you said at issue #319~, it would make it possible to re-use e tags instead of adding another one. Then clients would just need to change a little (adding all ancestors as e tags instead of just root and reply ones) and add the level/depth l tag. Markers could be there unchanged just for backward compatiblity. What do you think?

Edit: Either way, some e mentions aren't that bad and could be just filtered out client-side.

[mikedilger]

I think this issue is addressing a real need, but not one that comes up very often. So I support the effort. But I haven't taken the time to read this PR or the comments and I have no idea how to solve this. I'm just taking enough time to make this comment since you called for client authors.

[arthurfranca]

@mikedilger thank you for keeping an eye on this.

The tl;dr is have a way to 1) request replies to an event (not necessarily root one) n levels down on the tree of replies 2) request event ancestors n levels up on the tree.

The changes would be viable only if most client authors, like you, would be willing to add some tags when creating a kind 1 event. The least disruptive way (keeping backward-compatiblity while adding a minimum set of changes) I could come up with would be something like this example of a reply event:

{
    tags: [
      ['e', 'abc...', '<relay-url>', 'root'] // root event id
      ['e', 'def...', '<relay-url>'], // NEW - another ancestor, just like 'root' and 'reply' are ancestors of this event too ('ghi' is a reply to 'def')
      ['e', 'xyz...', '<relay-url>', 'mention'], // whatever mentions
      ['e', 'ghi...', '<relay-url>', 'reply'] // replied to event id
      ['l', '3'] // NEW - zero-based level/depth (in practice, just add up 1 to what was 'ghi' `l` tag OR count the non-mention e tags)
    ],
}

So just keeping all ancestors around inside e tags (not just root and reply) and adding an l tag to count the level/depth.

Edit: added the missing '\<relay-url>' part.

[monlovesmango]

this would require finding all the ancestors for any particular event in order to create a reply which adheres to this standard. since there are already many clients not adhering to the recommendation of adding relay hints (even the example directly above neglects to show relay hint placeholder) it is pretty common to not be able to find ancestor events. if we are creating a new standard to make querying replies easier I don't want it to rely on something that is not guaranteed to be findable on nostr.

In my mind the only scenario in which querying for direct replies is problematic currently is for replies to root events, as the relay will return all events in the thread rather than just direct replies. I would support a change that gives direct reply tags and root tags different generic tags, and any event that is a direct reply to a root would have both a direct reply tag and a root tag. that way you can query the replies to a root event either by direct replies or by the whole thread.

for all non-root events we already have the ability to query for direct replies. however if we start adding 'e' tags for all ancestors this functionality will break.

also 'r' tag is being used to stand for 'relay' in nip65. not sure if we want to re-use 'r' tag with a different meaning here.

generally i don't care about knowing the exact thread depth of any particular event or querying by thread depth. I just want to be able to query for root ancestor, direct ancestor, thread replies, and direct replies for any event. currently for non-root events you can accomplish all of these. however for root events you cannot query for direct replies (and ancestor queries don't apply).

[vitorpamplona]

l is not a good idea. Most client screens don't need to load the full thread. So, figuring l out for a new reply requires doing a lot more work. I am against it.

I am in favor of the practice of keeping all reply ids of the current branch of the thread (full path from leaf to root). Amethyst re-assembles the branch even if they are not there, but it's nice (faster) when they do. Current e tags are enough for that.

Reaction events (NIP-25) must copy b and l tags from the reacted-to event.

We should avoid this. Reactions need to be as lean as they can. It is just too much data to represent a "like".

An event can not be a reply to more than one event. (Or else it would mess with the l tag)

What if a reply from Thread A quotes another reply from thread B? I am in favor of differentiating "branch-path" and "citation"/"mention" e tags.

[arthurfranca]

@monlovesmango thank you for taking the time to give your view on it.

this would require finding all the ancestors for any particular event in order to create a reply

It would be enough to turn the previous id that has "reply" marker (or the last one, if deprecated way) into an ancestor e tag.

(even the example directly above neglects to show relay hint placeholder)

My bad. I will edit and fix it. Thanks for pointing it.

In my mind the only scenario in which querying for direct replies is problematic currently is for replies to root events

Yes it is the main problem and fixing this alone would be already a win. But some days after I created the issue, I found a recurrent use-case in major social apps (non-nostr) regarding what I called MITs, most interesting threads that would take advantage of requesting events at a specific level (sometimes deeper than direct descendant, so the l tag) and at a specific branch (that's why all ancestors would be needed).

for all non-root events we already have the ability to query for direct replies. however if we start adding 'e' tags for all ancestors this functionality will break

If i got it right, it wouldn't break if the ancestor e tags are not inserted at the last position ("deprecated" NIP-10 says the last one is the parent event). If using markers ("preferred" NIP-10) it would be there with the "reply" marker. Am I right?

also 'r' tag is being used to stand for 'relay' in nip65. not sure if we want to re-use 'r' tag with a different meaning here.

I think capital letters are allowed like 'R' instead

generally i don't care about knowing the exact thread depth of any particular event or querying by thread depth...

I understand that. If we make sure the change is backward-compatible, no clients would need to change the way they load threads, it would be just an option.

[monlovesmango]

@arthurfranca

It would be enough to turn the previous id that has "reply" marker (or the last one, if deprecated way) into an ancestor e tag.

if you do this you erode the ability to query for ONLY direct replies to the replied event's replied event. say you have event D that replies to event C (which is not the root event), and you want to reply to event D with event E. if you take event D's tag of event C and place it on event E as well, then querying for tags of event C no longer only returns direct replies but also replies of replies. the same problem we have with root events currently.

I found a recurrent use-case in major social apps (non-nostr) regarding what I called https://github.com/nostr-protocol/nips/issues/267#issuecomment-1450591924 that would take advantage of requesting events at a specific level (sometimes deeper than direct descendant, so the l tag) and at a specific branch (that's why all ancestors would be needed).

can you explain how querying for events at an arbitrary level makes it an interesting thread? comment, like, and zap counts definitely indicate whether a thread is interesting, but not level. if you use level you still have to query around for other pieces of data to truly determine if it is interesting. I am not grasping how querying by level lets you know its interesting and worth presenting to the user.

If i got it right, it wouldn't break if the ancestor e tags are not inserted at the last position ("deprecated" NIP-10 says the last one is the parent event). If using markers ("preferred" NIP-10) it would be there with the "reply" marker. Am I right?

no I don't think so. you cannot query a relay by 'e' tag position, nor 'e' tag marker. only by 'e' tag's second element, the event ID. when querying by 'e' tag, any event that has an 'e' tag (regardless of position or marker) will be returned.

I understand that. If we make sure the change is backward-compatible, no clients would need to change the way they load threads, it would be just an option.

if we don't have buy in from client authors to use this functionality to load threads then the whole incentive for implementing this is moot, regardless of whether they are willing to implement it in reply event tags. the LOE needed to implement this is still secondary to whether client authors find it useful.

I would support a branch 'b' tag to denote the thread branch a event is replying as to @vitorpamplona suggested with some stipulations:

1. the 'b' tag for the root event can be clearly determined (either by a marker or ordering or something else)
2. direct replies must still be tagged with 'e' tag (and can also be tagged with 'b' tag), otherwise we are just recreating the same problem we currently have of not being able to query for root event direct replies except with a whole new tag and for all events, not just root events.
3. the only mandatory 'b' tag is for the root event, all other 'b' tags are optional

[arthurfranca]

@vitorpamplona thanks for joining the discussion!

l is not a good idea. Most client screens don't need to load the full thread. So, figuring l out for a new reply requires doing a lot more work. I am against it.

Add 1 to the parent event's l tag value. (Note: Root event is considered l zero). Alternatively, the l value it is the number of ancestors (or know ancestors in case of non conforming event, so best effort in this case).

I am in favor of the practice of keeping all reply ids of the current branch of the thread (full path from leaf to root)

That would be an upgrade indeed. Although without the l tag, it wouldn't address what @monlovesmango said: "In my mind the only scenario in which querying for direct replies is problematic currently is for replies to root events".

We should avoid this. Reactions need to be as lean as they can. It is just too much data to represent a "like".

You are right. I'm ok with keeping this part out. Although it would be a way to request branch replies that were liked by original poster or by a friend, so a missing opportunity to enhance clients further.

"An event can not be a reply to more than one event" - What if a reply from Thread A quotes another reply from thread B? I am in favor of differentiating "branch-path" and "citation"/"mention" e tags.

Can't have two parents, or else the reply would be from 2 branches (would have to copy both branch ancestor tags). Quote/Mentions kind of get in the way and should be filtered out client-side when requesting reply events if we reuse the e tag, as already happens. My previous proposal was to have a new tag for branch event ids (the b or even capital A meaning ancestor), and keeping e just for mentions. The down-side is that current clients are already using e for "root" and "reply", which are also ancestors. Using a new tag would need even more adoption from client authors. I'm ok with both ways. You, client authors, should decide.

[monlovesmango]

That would be an upgrade indeed. Although without the l tag, it wouldn't address what @monlovesmango said: "In my mind the only scenario in which querying for direct replies is problematic currently is for replies to root events".

actually it would address this, as long as this full path leveraged the 'b' tag instead of 'e' tags. which is what I thought @vitorpamplona was suggesting.

'b' tags would be used exclusively for full thread searches. 'e' tags would be used for direct reply searches. but also need the stipulations I outlined at the end of this post

[arthurfranca]

[...] then querying for tags of event C no longer only returns direct replies but also replies of replies. the same problem we have with root events currently.

You make a good point. It does can be a breaking change for clients that currently do this (Are you currently relying on this? As you said, the problem already exists for root tags so it seems clients are loading all replies at once and then filtering client-side the slice they want, so maybe won't be a problem)

can you explain how querying for events at an arbitrary level makes it an interesting thread? comment, like, and zap counts definitely indicate whether a thread is interesting, but not level. [...]

It wouldn't be an arbitrary level. See how Twitter shows a branch A-B-C and A-X-Y at top if C and Y are from the same author than A? (It is an interesting thread/slice cause the original poster replied to it) So you request for replies 2 levels down A's level that has A's pubkey. This is one of many examples unlocked by the branch (where A is ancestor) and level/depth (n levels below A) tags.

Likes alone may be spammed, so that's why I also added that reaction events should copy all ancestor tags and the level tag from replied to event. (But @vitorpamplona don't like it because it makes reactions bigger). Knowing from what branch and level a zap is would be also good, but i haven't read zap NIP.

no I don't think so. you cannot query a relay by 'e' tag position, nor 'e' tag marker. only by 'e' tag's second element, the event ID. when querying by 'e' tag, any event that has an 'e' tag (regardless of position or marker) will be returned.

I think markers were a mistake because of what you said. I think not being able to query by relay url isn't a big deal, but many times you want just the events with a specific marker but you end up getting events with a different one. Ideally, e tags shoud be just for one use case such as for mentions.

actually it would address this, as long as this full path leveraged the 'b' tag instead of 'e' tags. which is what I thought @vitorpamplona was suggesting.

Yes you are right. I was assuming a new way of requesting direct descendant could be used (one level down, so request with event current's l value + 1). You are right, checking the reply marker and using a tag different from e would work. Although not for the Twitter example I gave you.

Any upgrade is a win. If you think allowing more use cases with the l tag isn't worth it I'm ok. Not my call as I'm not a client author yet. I just think it would be a small addition not that hard to support.

[arthurfranca]

I would support a branch 'b' tag to denote the thread branch a event is replying as to @vitorpamplona suggested with some stipulations: [...]

@monlovesmango So if I wasn't able to change your mind regarding l tag with my previous comment, an example reply event you suggest would be: 1) With markers (not sure if you want to keep e-root for backward compatibility for a while or not so i removed it):

{
    /* ..., */
    tags: [
      ['b', 'abc...', '<relay-url>', 'root'], // root ancestor id (no particular order)
      ['b', 'def...', '<relay-url>', 'ancestor'], // itermediate ancestor id (optional - no particular order, use created_at for sorting)
      ['e', 'xyz...', '<relay-url>', 'mention'], // mention id (no particular order)
      ['e', 'ghi...', '<relay-url>', 'reply'] // parent ancestor id (no particular order)
      ['b', 'ghi...', '<relay-url>', 'reply'] // parent ancestor id (no particular order)
    ]
}

2) With ordering, no markers

{
    /* ..., */
    tags: [
      ['b', 'abc...', '<relay-url>'], // root ancestor id (must be first b tag)
      ['b', 'def...', '<relay-url>'], // itermediate ancestor id (optional - below root and above the direct parent one)
      ['e', 'xyz...', '<relay-url>', 'mention'], // mention id
      ['e', 'ghi...', '<relay-url>', 'reply'], // parent ancestor id
      ['b', 'ghi...', '<relay-url>] // parent ancestor id (last b tag)
    ]
}

While @vitorpamplona (if still not buying the l tag need), which said "Current e tags are enough for that" and "I am in favor of differentiating "branch-path" and "citation"/"mention" e tags" wants:

{
    /* ..., */
    tags: [
      ['e', 'abc...', '<relay-url>', 'root'], // root ancestor id (no particular order)
      ['e', 'def...', '<relay-url>', 'ancestor'], // itermediate ancestor id (mandatory - no particular order, use created_at for sorting)
      ['e', 'xyz...', '<relay-url>', 'mention'], // mention id (no particular order)
      ['e', 'ghi...', '<relay-url>', 'reply'] (no particular order)
    ]
}

Sorry if i misunderstood. Now I ask you:

• How can the three ways converge?
• And have I convinced both of you of the importance of the l tag for future use cases and its easy of implementation?

[monlovesmango]

I like option 1 a lot. this would also enable doing thread-like queries for non-root events, which I think would be really convenient. the last 'b' tag with 'reply' marker should also be marked as optional. agree that 'e'-'root' tag should no longer be needed.

I know I suggested ordering, but would prefer to use markers over ordering as nip10 deprecated using ordering and want to stay consistent.

I am against option 3 with only 'e' tags as this will now make finding direct replies inefficient for non-root events as well.

Sorry, I am still not buying that 'l' tag is useful/needed. But would be nice to hear what other client devs think.

Thanks for pushing this conversation forward @arthurfranca !

[vitorpamplona]

To be clear, I don't have a use for l or for a filter for immediate replies on kind 1. Those who do should have a bigger say in this than me.

What about an additional filter attribute/format for any non-index-1 tag element:

  {
    "kinds": [1, 30023],
    "#e": [ 'abc...' ]
    "#e:3": [ 'reply' ]
    "limit": 5
  }

#e is assumed to be #e:1. You could filter by relay using #e:2 and by marker using #e:3.

All #e* tags are required to pass the tag filter together, so the last example reads:

Give me the latest 5 events 
where kind in [1, 30023] 
  AND at least one tag where ( tag[0] == "e" AND tag[1] in ['abc...'] AND tag[3] in ['reply'] )

You could download only kind 1s that directly cite a user:

  {
    "kinds": [1],
    "#p": [ 'abc...' ]
    "#p:3": [ 'mention' ]
  }

The same could happen for p, where #p is assumed to be #p:1 and #p:2 is a relay filter for people, and #p:3 is a nickname filter.

Then all tags should be indexed. You could even do a search by delegation token using #delegation:3 on NIP-26,

  {
    "kinds": [1],
    "#delegation:3": [ "6f44d7f...e5f524"]
   ]
  }

or do a search for nudity reports using #e:2 in kind 1984.

  {
    "kinds": [1984],
    "#e:2": [ "nudity" ]
  }

What about a search for image sizes on NIP-58?

  {
    "kinds": [30009],
    "#image:2": [ "1024x1024" ]
  }

The format could be #<tag name>:<array index>

I assume this could be beneficial for other event kinds that have more information on them.

[monlovesmango]

relay devs would have to weigh in on that, as it means that more than just the first two elements of a tag need to be indexed in the relay db. if people are ok going down this route then we can probably get by with just 'e' tags.

personally I do see the appeal of having these kinds of filters, however I worry more about bloating the relay db. indexing the 3rd element provides significantly less value than indexing the 2nd element, but they both have the same cost. also there are no essential use cases that need anything but the first 2 elements of a tag for querying, and all of the potential use cases you highlighted above are bit weird and made up (except the first one that this new type of filter would solely be created for).

[arthurfranca]

@monlovesmango – I like option 1 a lot. this would also enable doing thread-like queries for non-root events [...] @vitorpamplona – I am in favor of the practice of keeping all reply ids of the current branch of the thread (full path from leaf to root). Amethyst re-assembles the branch even if they are not there, but it's nice (faster) when they do. Current e tags are enough for that.

@monlovesmango @vitorpamplona I see we can settle that both of you liked having all ancestor ids listed, very useful for walking up the tree faster among other uses. We effectively moved away from having a parent tag. Though we still have to solve the problem of immediate replies for root events.

@staab – I like this solution a lot. Replies are the one universal concept in social media applications, event tags are far too broad to support them efficiently. @barkyq – I like this too. In general, would be nice to have efficient ways to build reply trees. @mikedilger – I think this issue is addressing a real need, but not one that comes up very often [...] @monlovesmango – In my mind the only scenario in which querying for direct replies is problematic currently is for replies to root events

Although @staab, @monlovesmango, @mikedilger, @barkyq and I consider this part important, @vitorpamplona doesn't care much for his client as he said "I don't have a use for l or for a filter for immediate replies on kind 1". Yet, @vitorpamplona suggests a way of solving it: "an additional filter attribute/format for any non-index-1 tag element", which is ingenious, but would certainly have to be a new NIP (instead of NIP-10 edit, keeping an eye on breaking changes as @fiatjaf is worried), with a risk of less adoption cause it would also depend on relay operators. As @monlovesmango says, "relay devs would have to weigh in on that [...]".

So, as far as I know, for root immediate replies we have this novel filter (needs new relay functionality)...

{
    "kinds": [1],
    "#e": [ 'root id' ]
    "#e:3": [ 'reply' ]
    "limit": 5
}

... versus this filter (need the l tag your guys don't like, but would solve it among other uses)

{
    "kinds": [1, 30023],
    "#b": [ 'root id' ], /* I used #b instead of #e cause of @monlovesmango*/
    "#l": [ '1' ]
    "limit": 5
}

[vitorpamplona]

I think the main question is:

Are markers good? Are markers a good design pattern for new NIPs moving forward? Do we want more e tags with a classifier or not?

If so, we should have extended filters for them.

If not, we should expect, and work through all event kinds, to avoid the use of generic tags like e and p in favor of kind-specific tags, such as branch, reply, root as tag names for kind 1.

[vitorpamplona]

Here are my 2 cents:

e and p are very useful generalizations for the event/user graph. e and p are, in my mind, fundamental parts of the protocol and not a Kind 1 element. When relays and clients see e and p tags they can generally assume the next element is the hex and index it. There is no need to create base-graph-named indexes for each event kind out there. New event kinds can reuse indexing if they simply follow this naming convention.

d and a tags messed things around a bit, but they are still generalizable and indexable, independent of event kind. As long as the d tag stays a pub-key-based unique id and the a tag remains a reference to that id, it should be fine.

r, t, and g tags are already more domain (event kind) specific.

By moving to a b tag, we would lose that automatic indexing element and opt for a kind-1-only tag, disabling such node connections if a relay only processes e tags.

IMHO, It would be better to keep using e with markers so that the event graph is maintained. Once that is established, it's natural to suggest a filter by a marker (indexed or not). If we always assume such queries to not be indexed, then it's natural to suggest a filter by any "column" in tags.

By having everything filterable, there is less of a need to create new tag names when simple extensions to the graph are required by domain-specific needs of each event kind out there.

For instance, I would love to have the same direct mention/citation marker for r, t, and g tags. When things are directly cited in a message, they take a whole different meaning at the client level. I don't think we should be making mentioned-r, mentioned-t, and mentioned-g tags.

[arthurfranca]

When relays and clients see e and p tags they can generally assume the next element is the hex and index it. New event kinds can reuse indexing if they simply follow this naming convention.

I think relays aren't applying special treatment depending if the tag is e or z. Well, atleast up until now it seems care was taken so that a tag with same char has the same possible array of value types no matter what event kind. But I wouldn't count on that as searchable tags only have 1 char, so a little limited set.

---

~Today the problem with markers is that relays won't filter by them. As a result, clients must fetch a possibly much bigger number of events than would otherwise be needed.~

~So, as things are today, markers maybe were not the best idea, because the data inside an e marker is important, not secondary. e started as meaning "reply event" because nostr was at its infancy. But now it can mean "root event", "reply event", "mention event" and the possible meanings (markers) may keep expanding, and these aren't filterable when requesting.~

~Currently, using different 1-char tags is the way to differentiate things and so using e for different contexts doesn't work very well. But it is not the end of the world while there are only 3 meanings. Cause root and reply can be merged with the concept of ancestors (if using what I already proposed) and then filtering out cliet-side the mentions, that are just a few in a thread (and commit to not adding new markers).~ Edit: Considering the proposed solution for the specific use cases on the current open issue, markers are not a big problem. Perhaps it will become a problem if new meanings for event references on text notes arise.

Expanding all tag values (not just the value at position 1) to be searchable may rule out some relay databases, decreasing db diversity. Perhaps it is easier to squeeze the current nostr-protocol further, as it already demonstrates it is capable of enabling many use cases without change. I really don't know.

[arthurfranca]

I'm going to illustrate this issue with screenshots to try pushing this forward. Hopefully I will undoubtedly demonstrate how treating replies as a tree (branch and level/depth) will allow nostr clients to present themselves as real contenders –think it as real adoption.

First, see below how Youtube push an old comment to the top just because the OP liked it. It will go all the way up above any spam! Great, uh?

Example nostr filter:

  {
    "authors": [OP pubkey],
    "kinds": [7], /* reaction */
    "#e": ['root event id']
    "#l": ['1'],
    "limit": 5
  }

It works with events other than the root one too, just change #e and #l. Note: it would need reaction to copy all b (or e if we reuse them instead) and l tags from the reacted to event

[arthurfranca]

Now we can see how Instagram push direct replies to the top if it was replied by the OP. All the way above any spam! Great again!

Example nostr filter:

 {
    "authors": [OP pubkey],
    "kinds": [1],
    "#e": ['root event id']
    "#l": ['2'],
    "limit": 5
  }

Easy peasy! (But not with current NIP-10 👎 )

[arthurfranca]

Now featuring @fiatjaf himself! You can see the same pattern on Twitter! How the direct reply is shown at the top of the thread, beacause it was replied by fiatjaf, the OP:

[arthurfranca]

Above screenshots show how we could load most "interesting" replies at the top, while keeping spam and least interesting replies at the bottom.

All apps shown also benefit from lazy-loading the thread instead of loading everything at once. It is just not a best practice requesting all thread at once.

All this would be possible with the changes I'm proposing. Changes to kind 1 creation that aren't complex at all.

[fiatjaf]

I personally think these screenshots show some of the worst aspects of these centralized social platforms. The UX is optimized for useless pumping of engagement and awful for actual conversations, it's a lab rat. So far Nostr clients have been delivering an experience that is far superior to the annoyances depicted above.

[arthurfranca]

I personally think these screenshots show some of the worst aspects of these centralized social platforms. The UX is optimized for useless pumping of engagement and awful for actual conversations, it's a lab rat. [...]

Omg, I consider it is totally the opposite ^-^. I understand this is debatable. Also, you created Nostr so you get to decide what gets merged.

A person reading a post by Alice will want to see comments that Alice replied to, because she started the coversation. If I'm reading Alice's blog, I want to know her opinion. I want to follow conversations in which she is involved. Of course not exclusively, but it has a higher probablity of being more important than a Random guy's comment that could be spam or a sub thread going totally out of topic. I also consider it a good way to push down spam.

Likes are usually evil, but if we promote to the top the comments liked by your friends, I guess now they are not anymore. In the example I gave, it was a like from OP, but could be from your friends.

If we are talking about a client as Twitter that limits char count, I also want to read the thread of Alice replying to herself multiple times first.

[...] So far Nostr clients have been delivering an experience that is far superior to the annoyances depicted above.

If using paid relays, paid with satoshis, that regular Joes can't use to join conversation. Haven't heard of any relay that fights spam by using another technique yet. What I presented at least may allow clients to hide spam below comments that client at least consider on topic.

Lets not forget the cell phone plan spending GB quota faster than it could be by downloading entire threads from multiple relays. We need lazy-loading possibility in my opinion.

[arthurfranca]

Just as a side note: thinking about it, there is no strong reason to need a branch b tag.

It works ok with regular e and/or a, cause as subscription filters will be relative to known event at hand, we know if it is a text note or a long-form content. I will edit the screenshot filters above accordingly.

In a nutshell, a new reply event just needs to: 1) copy all e and a from replied to event that aren't mentions (removing reply marker if present), so to keep all ancestor list – instead of just the root as current clients do 2) append the replied to event's id/address as e/a tag with root or reply marker – as already is done by current clients 3) count the resulting e and a entries that are not mentions and add as an l tag ([l]evel or capital D for [D]epth)

But yeah, if no developer other than me gets the importance of this, this will unfortunately never see the light of the day.

[arthurfranca]

@vitorpamplona I've downloaded Amethyst and noticed it splits feed into two tabs: "New topics" and "Threads" (don't know exact text as mine is in pt-BR).

In the future, if clients adopted the level/depth tag, including even on root events (['l', '0']), you would be able to fetch each tab content separately, just when user touches it. It would be more efficient.

It would allow you to get only root events with the filter { kinds: [1], authors: [...], #l: ['0'], limit: 20 }

And also the other tab with the following filter { kinds: [1], authors: [...], #l: ['1', '2', '3', '4'], limit: 20 }

[vitorpamplona]

The issue is that I never expect ALL clients to adopt this mode. Thus, to avoid UX inconsistency with posts that should have been there but are not, I most likely would lever use this as a filter. I would download everything and "fix stuff up" when they arrive.

[arthurfranca]

Yes many clients will want to support all the known patterns of replying to an event to not miss events when fetching them. Atleast for a transition period. For example, I don't expect the "deprecated" section of NIP-10 to be supported forever by clients, specially new ones.

I wanted to point this out to show the usefulness of the proposed changes, among other uses already mentioned.

This issue is aiming to reduce number of events retrieved on some event subscriptions, while unlocking a few new use cases. If it makes it into a NIP, I expect that eventually up-to-date or new clients will adopt.