Reactions are inefficient. There needs to be an aggregate kind.

[jonathanleger]

Nostr needs a better way to handle likes/reactions. Nip 25 is a terribly inefficient way to do it. Burying reactions in the tags requires clients to do a huge amount of data gathering to properly show likes/reactions. Having an event kind/# that represents a tally of all likes for an event would be far more efficient, otherwise the client has to gather potentially thousands of events just to count up the likes for a note. I'll be creating a separate db table that mines and sums the reactions, but that creates a centralized solution rather than a decentralized one.

[mikedilger]

While I understand the efficiency problem, and tallying them would massively alleviate it, it breaks the trust model because clients don't trust relays.

Now, how many upvotes there are isn't something that needs high trust, IMHO, unless you want to. So here's my slightly modified proposal: We leave NIP-25 and clients continue to generate NIP-25 events, but clients that don't want to be flooded with individual reactions can instead query the relays for a total. If they instead were paranoid, they could query the kind 7 events directly and total them up for themselves.

AND while we are on the topic, if we are doing to do that for the reactions performance problem, we should also do it for the "how many followers do I have" performance problem.

My point being, these would be ADDITIONAL ways to get the data in a LESS trustworthy but much more efficient manner.

The other fly in the ointment is that no relay actually has all the data. So clients will need to somehow merge totals from multiple relays, and in doing so not double-count. That might require some advanced algebra that I'm not aware of.

[mikedilger]

Just as a data point, twitter does 3 million transactions per second to calculate and display the view count, and they can centralize and aggregate: https://www.youtube.com/watch?v=2tjCPXE0_Ko

[lidstrom83]

This service seems more appropriate at the user level because aggregation at the relay level just introduces the problem of reconciliation of aggregates across relays. Rather than anoint relays with this privilege/burden, anybody could instead publish a replaceable "aggregation event" as a reply to an event. Clients could subscribe to an aggregator like any other user and for any event they've replied to with an aggregation event, the aggregation event could be used to populate the reaction counts. Subscribing to multiple aggregators could help to keep them honest because significant disagreement would raise an alarm and it's easy to audit their work.

[eskema]

This service seems more appropriate at the user level because aggregation at the relay level just introduces the problem of reconciliation of aggregates across relays. Rather than anoint relays with this privilege/burden, anybody could instead publish a replaceable "aggregation event" as a reply to an event. Clients could subscribe to an aggregator like any other user and for any event they've replied to with an aggregation event, the aggregation event could be used to populate the reaction counts. Subscribing to multiple aggregators could help to keep them honest because significant disagreement would raise an alarm and it's easy to audit their work.

aggregations at the user event level is useless because it's probably outdated once you get them. It's better to request a COUNT from a relay, that way it's always up to date and doesn't occupy unnecessary space. of course, that client should store and compare / merge those counts from multiple relays and update it's own count number and display that. I wouldn't trust numbers from a pubkey except maybe some specialized ones that did that as a service, but they'd be fetching that data from somewhere somehow, and the best way to do that would be with a COUNT from relays.

[jb55]

there could also be a NIP that implements getting stats for a set of ids via REST:

POST ids -> /stats

[ {id: "abcd...", replies: 12, reactions: 100, reports: 100, quoted_boosts: 10},
  ...
]

[jb55]

this could be very efficient and could aggregate many relays, have caching, etc.

[jonathanleger]

there could also be a NIP that implements getting stats for a set of ids via REST:

POST ids -> /stats

[ {id: "abcd...", replies: 12, reactions: 100, reports: 100, quoted_boosts: 10},
  ...
]

+1 Even better

[lidstrom83]

aggregations at the user event level is useless because it's probably outdated once you get them

The idea is that the aggregation event is replaceable (NIP-33) and periodically updated by the aggregator who receives all reaction events from all relays.

I wouldn't trust numbers from a pubkey except maybe some specialized ones that did that as a service

The expectation is that a set of trustworthy aggregators would emerge because they effectively keep each other honest.

there could also be a NIP that implements getting stats for a set of ids via REST

Isn't nostr the natural place to publish these results? As an aggregator, I'd prefer to not have to run a web server, and nostr relays are already set up to do the work of serving this kind of data. Clients would also automatically get updated stats over the websocket which seems nice.

[tmathews]

I would like it if there was a solution to provide information over the existing websocket rather than having to additionally using REST as it adds more complexity (even tho pretty simple) into my application.

My naive approach would be to have additional properties added onto the event JSON which can be ignored when doing verification. If the server already has the data in memory (or a fast data store) it doesn't take too much to simply injected the extra data points which could have their own NIPs.

Another approach would be a new kind of REQ method for data points that don't follow the event structure and are purely for servicing means.

Aggregation data changes constantly so HTTP caching would be temporary at most. Still facing all the same problems described here, but allows the developer to continue working off the existing API approach. But perhaps this breaks things and I don't know what I'm talking about as my studying on this is pretty minimal.

[fiatjaf]

My naive approach would be to have additional properties added onto the event JSON which can be ignored when doing verification. If the server already has the data in memory (or a fast data store) it doesn't take too much to simply injected the extra data points which could have their own NIPs.

This is my preferred approach.

[jb55]

yes this is fine

[cameri]

My naive approach would be to have additional properties added onto the event JSON which can be ignored when doing verification. If the server already has the data in memory (or a fast data store) it doesn't take too much to simply injected the extra data points which could have their own NIPs.

This is my preferred approach.

@fiatjaf Do we need a NIP to do this? Or can we agree on a field name for tallies? Also, should relays always include rallies or should there be a filter for this?

[Giszmo]

@Cameri Of course you can implement it without a NIP first. Clients might get confused by extra fields in events though. To then use that field, we definitely should nip it. Nips are how we agree on things. I would only add the extra field on events that have reactions and as it's not for free to add this, you probably want to control it somehow. Some way of either switching it on for a connection or a new command like REQ_WITH_REACTIONS ...

@lidstrom83 while tallying by non-relays would be nice, it's very costly. Such a service would have to send an event for every :+1: and using nip33, the relay would save disk space but pay for it with CPU and disk i/o. Replacing events is relatively expensive.

[cameri]

@Cameri Of course you can implement it without a NIP first. Clients might get confused by extra fields in events though. To then use that field, we definitely should nip it. Nips are how we agree on things. I would only add the extra field on events that have reactions and as it's not for free to add this, you probably want to control it somehow. Some way of either switching it on for a connection or a new command like REQ_WITH_REACTIONS ...

@lidstrom83 while tallying by non-relays would be nice, it's very costly. Such a service would have to send an event for every :+1: and using nip33, the relay would save disk space but pay for it with CPU and disk i/o. Replacing events is relatively expensive.

{
  ...
  tallies: {
    "❤️": 69,
    "🤙": 420 
  },
  ...
}

[eskema]

these tallies, are they global? like, is the relay counting from all events it has? or is this derived from the request?

{ ... tallies: { "❤️": 69, "🤙": 420 }, ... }

these tallies, are they global? like, is the relay counting from all events it has? or is this derived from the request?

[cameri]

these tallies, are they global? like, is the relay counting from all events it has? or is this derived from the request?

{

...

tallies: {

"❤️": 69,

"🤙": 420 

},

...

}

these tallies, are they global? like, is the relay counting from all events it has? or is this derived from the request?

Not sure what you mean by global. This is an example of how tallies could look like on an event.

[eskema]

Not sure what you mean by global.

those numbers are coming from all reactions the relay has seen or from a list of authors on the request?

[cameri]

Not sure what you mean by global.

those numbers are coming from all reactions the relay has seen or from a list of authors on the request?

They came from me typing them. We don't have any of this implemented anywhere nor do I know the answer on how it's going to work.

[jb55]

new aggregation requirement: zaps with amounts counted from the bolt11. not sure if there's a general way to implement this, I will need custom magic regardless it seems like. especially since these are only counted if the zaps come from a zapper pubkey associated with a user's lnurl. for now I will continue pulling all of the zaps and count them client side.

[fiatjaf]

What are zaps?

[cameri]

new aggregation requirement: zaps with amounts counted from the bolt11. not sure if there's a general way to implement this, I will need custom magic regardless it seems like. especially since these are only counted if the zaps come from a zapper pubkey associated with a user's lnurl. for now I will continue pulling all of the zaps and count them client side.

What does a single zap event look like?

[fiatjaf]

@Cameri how feasible are these tallies for you to implement? Would you keep a separate cache for them somewhere?

@scsibug @atdixon how do you see this?

[cameri]

@Cameri how feasible are these tallies for you to implement? Would you keep a separate cache for them somewhere?

@scsibug @atdixon how do you see this?

I can cache reactions as they come in using redis and include them as events go out. But if clients don't request the event they won't see updates, so part of the deduplication logic would be to keep the highest tallies or something. Maybe we need to include a timestamp of the tally?

[cameri]

@Cameri how feasible are these tallies for you to implement? Would you keep a separate cache for them somewhere?

@scsibug @atdixon how do you see this?

I can cache reactions as they come in using redis and include them as events go out. But if clients don't request the event they won't see updates, so part of the deduplication logic would be to keep the highest tallies or something. Maybe we need to include a timestamp of the tally?

We could also just send unsigned events with the tallies.

[jb55]

What are zaps?

the kind 9735 thing I posted in telegram awhile back. still need to write it up but damus and snort already implement it.

[atdixon]

Serving each individual like is definitely not great / ultimately untenable. So some aggregation solution is going to be warranted.

In any case, I'd make it a separate request (instead of tacking onto events) so that tallies can be refreshed w/o re-fetching events, and separate band keeps things simple.

But - controversial opinion maybe? - I don't think relays should take this on or have it in nips/spec. There are too many issues ... how does a client sensibly merge tallies across relays? what do "paranoid" clients do...the most skeptical tallies are going to be high-volume reactions...so does client go back and request them all? As far as implementation goes, it's not trivial...counters always suck and here you have to de-dupe extra likes from the same pubkeys? ...

Why not consider this a 3rd party provider thing? A simple focused tally service and given clients can integrate with their preferred service choice and forward on tips or what-have-you. (Think of tally service more like search service this way, not something in spec/nips ... or if these are, make them separate specs from relay specs and not websockets, which are not fitted to search-me-a-thing or refresh-me-some-reaction-tallies.)

[mikedilger]

Not sure what you mean by global.

those numbers are coming from all reactions the relay has seen or from a list of authors on the request?

I think people want to see a count of all reactions on the relay, not filtered by the filter except insomuch as being applied to the event which is filtered by the filter. But that's just what I think.

[eskema]

Not sure what you mean by global.

those numbers are coming from all reactions the relay has seen or from a list of authors on the request?

I think people want to see a count of all reactions on the relay, not filtered by the filter except insomuch as being applied to the event which is filtered by the filter. But that's just what I think.

in that case, that feature will be useless (for me at least) once people simply autobot those counts with multiple pubkey farms. I don't think it should be encouraged to return such counts without beeing given some filter, but that's just my opinion of course.. I understand that it would alleviate much pains... maybe I need to rethink how reactions are meant to be used if that's the way it's supposed to work.

[mikedilger]

how does a client sensibly merge tallies across relays? what do "paranoid" clients do.

It might be more data than simple counts, but something like this could be merged across relays and is still less data than pulling each reaction event. I'm not sure about it, I'm just throwing it out there.

[
  "❤️": ["pubkey1", "pubkey2", ..., "pubkeyN"]
  "🤙": ["more","pub","keys","that","are only","valid if in","just one","reaction"]
]

[eskema]

how does a client sensibly merge tallies across relays? what do "paranoid" clients do.

It might be more data than simple counts, but something like this could be merged across relays and is still less data than pulling each reaction event. I'm not sure about it, I'm just throwing it out there.

[
  "❤️": ["pubkey1", "pubkey2", ..., "pubkeyN"]
  "🤙": ["more","pub","keys","that","are only","valid if in","just one","reaction"]
]

this would return multiple arrays of millions of pubkeys at some point

[cameri]

how does a client sensibly merge tallies across relays? what do "paranoid" clients do.

It might be more data than simple counts, but something like this could be merged across relays and is still less data than pulling each reaction event. I'm not sure about it, I'm just throwing it out there.

[
  "❤️": ["pubkey1", "pubkey2", ..., "pubkeyN"]
  "🤙": ["more","pub","keys","that","are only","valid if in","just one","reaction"]
]

this would return multiple arrays of millions of pubkeys at some point

tags: [
  ["c", "❤️", "100", "pubkey1", "pubkey2", ..., "pubkeyN"],
  ["c", "🤙", "45", "pub","keys","that","are only","valid if in","just one","reaction"]
]

c: count first element: reaction second element: count third and on: pubkeys, not all need to be included

[mikedilger]

tags: [
  ["c", "❤️", "100", "pubkey1", "pubkey2", ..., "pubkeyN"],
  ["c", "🤙", "45", "pub","keys","that","are only","valid if in","just one","reaction"]
]

c: count first element: reaction second element: count third and on: pubkeys, not all need to be included

Except that tags are part of the author-signed data.

[cameri]

tags: [
  ["c", "❤️", "100", "pubkey1", "pubkey2", ..., "pubkeyN"],
  ["c", "🤙", "45", "pub","keys","that","are only","valid if in","just one","reaction"]
]

c: count first element: reaction second element: count third and on: pubkeys, not all need to be included

Except that tags are part of the author-signed data.

Indeed. This works if the tallies are sent as a separate event kind sent by the relay. Just throwing things out there.

[fiatjaf]

Why not consider this a 3rd party provider thing? A simple focused tally service and given clients can integrate with their preferred service choice and forward on tips or what-have-you. (Think of tally service more like search service this way, not something in spec/nips ... or if these are, make them separate specs from relay specs and not websockets, which are not fitted to search-me-a-thing or refresh-me-some-reaction-tallies.)

I like this.

I would like even more if clients stop caring about these likes, since they are meaningless and so easy to fake -- unless clients only wanted to see likes from some specific reputable pubkeys, then it makes more sense.

Lightning tips are a better idea, as they are not completely meaningless, and at least the receiver can be sure that they were real, not fake.

[mikedilger]

Lightning tips are a better idea, as they are not completely meaningless, and at least the receiver can be sure that they were real, not fake.

I think I could tip myself over and over, transferring the bitcoin around in a circle.

I like the other idea better: stop caring about these likes.

[gkbrk]

You don't need to do anything special to keep extra JSON fields out of the signature verification, because the fields used for calculating the event ID are already specified. If clients get confused by extra fields, the broken clients should probably be fixed.

I noticed some clients and relays already make use of extra event fields that are not included in the signature. When reposting from another relay, some apps include the URL of the original relay. I haven't seen apps or relays that break from this though.

[pjv]

Not trying to be glib at all here - sincere fundamental questions:

Does nostr need reactions at all?

Why do we care who or how many “liked” something?

Are “likes” (reactions) part of the whole mind parasite addiction game invented by conventional social media “platforms” that nostr might be the antidote to?

What if nostr let one person make their thoughts / words / images available to others without the endlessly gamed and game-able metadata. Then those others are entirely free to make of those thoughts / words / images whatever they will without the subtle or not-so-subtle influence of an embedded middle-school popularity contest (the only purpose of which is to inform people how they “should” feel about something)?

Personally, I think the more bare and spare that nostr (the protocol) is, the more likely it is to actually work (technically and otherwise).

LOL - go ahead and rain down the unlikes…

[Giszmo]

Nips are not about defining what all the clients have to support. It's about making clients interoperable and some definitely want "likes", so you not wanting them won't make "likes" go away.

Sadly, social media competes with social media and skipping on the dopamine trick won't help to move people over. As clients within nostr are also competing for the more mobile users (switching between Twitter and nostr is harder than between Astral and Snort), I guess most popular nostr clients will at least optionally support "likes".

[fiatjaf]

@pjv I agree with you. I dislike reactions myself too, but the NIPs exist to standardize stuff so clients are kept compatible. A lot of people unfortunately like reactions, so why not let them have them in a standardized manner?

There are multiple clients that do not show reactions.

[cameri]

@pjv I agree with you. I dislike reactions myself too, but the NIPs exist to standardize stuff so clients are kept compatible. A lot of people unfortunately like reactions, so why not let them have them in a standardized manner?

There are multiple clients that do not show reactions.

I think we have to think a bit bigger, likes are just one use-case. I think we should think in terms of tallies.

Now that clients are sending blocks/mutes, relays could include those tallies up as well.

So what we need is a way to aggregate any event and a way to fetch for those aggregates.

You can still query a relay to see what the actual events used for the tallies are.

[mikedilger]

the only purpose of which is to inform people how they “should” feel about something

@pjv The only reason I ever like a post is to tell the poster I liked it, to give them positive feedback, as a shortcut to saying "I concur". I don't care about the tallies. I've been convinced that it would be better if I actually replied with an emoji instead, so I'm trying to remember to do that. I made reactions optional in gossip.

[gkbrk]

Now that clients are sending blocks/mutes, relays could include those tallies up as well. So what we need is a way to aggregate any event and a way to fetch for those aggregates.

Especially for blocking and muting, you might need some advanced filters instead of just blind tallying. After all a single user in Nostr can generate 10000 block events in order to skew the tallied counts.

I'm guessing "count block events by friend-of-friends" will be a heavy query for relays to do automatically. Maybe if your social graph is small enough to fit in a nostr filter it might work with a simple count.

[eskema]

I use reactions the same way as any other kind-1 reply, I treat them as replies with a single char and hide away the option to reply to them, but the whole logic is more or less the same to me, might make some cosmetic changes but that's it. I like reactions because they're a simple quick and dirty acknowledgments and are good for discoverability as some people are more prone to simply react than to reply. some people don't like them, that's fine, they don't need to use them and they can hide them completely if wanted. any pre-made tallies that do not come from a filter provided by a user request is simply promoting the use of spam metrics and is encouraging the same behaviour that makes current social media a dumpster fire.

[pjv]

@mikedilger @fiatjaf @Giszmo @eskema

I probably shouldn’t have started blabbing in here. I’m not (yet?) building anything on nostr (just a user and small relay operator (running nostream, @Cameri - thanks!)) and probably don’t know what I’m talking about.

I really wasn’t intending to say “I don’t like likes so let’s not have them” though I can see how what I wrote might have come off that way.

I do think that reactions - as they are used in legacy social media - are an anti-feature with respect to authenticity. I don’t think this is the right place for a treatise so I’ll spare you all a giant, esoteric rant. Obviously nostr is positioned as an alternative to present-day SM platforms and really bravo - what you are building seems like it has so much promise.

It seems super-important here at the beginning to carefully consider which aspects of legacy social media it is best to emulate (because there is plenty of well-worked-out UX out there) and which it is best to leave behind. “Best” not in the sense of what makes the most compelling twitter clone (and I know that nostr is the basis for way more than that - just picking on this microcosm as a representation for the macro), and also not necessarily “best” in the sense of what people think they want (because nobody wanted an iphone before there was one), but instead “best” in the sense of what adds the most real value.

Again, I probably don’t know wtf I’m talking about here. I do devops for medium-sized (tens of thousands of sessions/day) e-commerce websites. I’ve never written code meant to run in a globally distributed / decentralized architecture. But I do think about scaling and I don’t see how nostr can stay decentralized and scale to millions, then tens of millions, then hundreds of millions of concurrent users without the relays being super-simple, commodifiable, and ALL over the place. And in that case, unless someone comes up with some real magic, the numbers next to the little heart icons (any of the aggregate numbers really) are always going to be lies.

Right?

If that’s right, then what is being discussed in here (and maybe also the whole NIP-25 / kind 7 concept) seems like something that’s likely to bog and bloat relays and clients in order to be able to show users fake news. And the rationale I think I’m reading here for showing them that fake news seems like it boils down to “people want fake news so we should provide it.”

Do I have that right?

If I understand what @eskema wrote above (and also I think what @mikedilger was saying), I think they’re exactly right; a reaction should be a kind 1 reply event. And nobody should be pretending to count anything.

[fiatjaf]

I don't think they will always necessarily be lies, but there will be trust involved.

My suggestion on this topic a long time ago was that these reactions should never be really downloaded as normal events (because bandwidth, efficiency etc) but always as counts, and also that some relays would want to include an extra field in the event body, like {"id", "...", ..., "likes": 99} and the client would choose one or two of these clients to show the counts from. These would be clients that have some form of sybil protection, so the like numbers would be somewhat meaningful.

[gkbrk]

My suggestion on this topic a long time ago was that these reactions should never be really downloaded as normal events (because bandwidth, efficiency etc) but always as counts.

This is the opposite of what I found useful with reactions. Rather than a meaningless and game-able count, the events themselves are more useful. But instead of downloading "all reaction events for event ID X" which includes a ton of spam and unrelated content, I use "all the reaction events of X, Y, Z pubkeys" where X, Y, and Z are people I follow.

In this context, it is useful to download the reaction events and see what my friends like and dislike. A count of N people liked this event is not helpful for me at all.

[pjv]

I don't think they will always necessarily be lies, but there will be trust involved.

I only meant “lies” in the sense that I don’t think it’s technically possible to actually compile accurate counts across a meaningfully decentralized and distributed network of dumb relays (though maybe I’m wrong in that assumption). But if someone looks at a post and there is a heart icon with an integer next to it, given (centralized platform) history (or probably even without that history), they are going to assume that integer is the actual number of people who “liked” on that post. Even aside from people and bots deliberately gaming, there’s not really any performant way to tabulate all the likes across all the relays (unless there were very few relays), is there? Maybe “lie” is too strong a word, but in the best of all worlds that number could only be a best guess that would likely have more and less slop in it based on a lot of unknowable variables.

Sorry for my noise. I’m going to try to shut up now and let you all get back to building nostr.

[fiatjaf]

It is not noise, I think you are raising important points that should be raised.

[fiatjaf]

I use "all the reaction events of X, Y, Z pubkeys" where X, Y, and Z are people I follow.

This is similar in spirit to what I suggested.

[lidstrom83]

I tend to agree with the reservations around a global "likes" count, but because people are doing this already, we can at least reduce the burden on relays. So I'm focusing on the question of "can we do this?" and punting on "should we do this?".

@fiatjaf's proposal doesn't appear to solve the problem of aggregating reactions across multiple relays (deduplication). Am I correct in assuming this is a problem we want to solve?

@lidstrom83 while tallying by non-relays would be nice, it's very costly. Such a service would have to send an event for every +1 and using nip33, the relay would save disk space but pay for it with CPU and disk i/o. Replacing events is relatively expensive.

A user only really cares about the first two digits in the tally, so aggregation events only need to be replaced when the count goes from e.g. 1.1k to 1.2k. This would ensure that the relative disk burden of these replacements quickly tends to zero at scale. I assume the benefit to the relay of not having to serve individual likes far outweighs this, but this may not be true for events with few reaction counts and views. In this case tallies could be started only after some threshold count of, say 10.

Another nice thing about the approach I've described in this thread is it allows for easy experimentation in approaches to filtering out fake "likes" - neither relays nor clients need be changed in order to try something new. E.g. one aggregator could try a reputation-based approach. Another one could filter by orange checks. Or a combination of these approaches. Or no filtering at all. The user could make the choice themself.

I'd be willing to make a proof of concept and a NIP if there's any interest in this approach. It'd be a kind 10007 note with content like @Cameri mentioned: { "❤️": 69, "🤙": 420 }.

If it's not clear, a client could then, for a given note, check whether their chosen aggregator has posted one of these notes in reply. If so, they'd use it to update the display rather than loading individual reactions.