hasura / graphql-engine

Blazing fast, instant realtime GraphQL APIs on your DB with fine grained access control, also trigger webhooks on database events.
https://hasura.io
Apache License 2.0
31.17k stars 2.77k forks source link

Feature Request: Event-based subscriptions #2317

Open heyrict opened 5 years ago

heyrict commented 5 years ago

Reopen of #1355 .

To inspect whether it is good to use event-based subscriptions or live queries, I've gone through many articles and found this comment best describes the current state of live queries and event-based subscriptions.

There are still use cases of event-based subscriptions, especially when performance count more than reliability (take an example of twitter like subscription)

Another thing is that, most subscriptions in the current graphql ecosystem use event-based ones. It will be hacky to convert an existing project with subscriptions to hasura.

The hacky part of using live query for event-based subscriptions is that

  1. It will be difficult or at least not straight-forward to detect row deletion. #1201
  2. It will yield unnecessary initial results.
  3. Tracking update or create events need extra effort (adding updated_at columns to sort with, etc.)

cc: @coco98

heyrict commented 5 years ago

Here is a cleaner workaround to have event-based subscription, though it needs another graphql server instance to expose the subscription API. See heyrict/hasura-trigger-subscription-server.

0x777 commented 5 years ago

@heyrict One of the biggest challenges/decisions with event based subscriptions is how does the client recover from loss of events? Does the server store what events are delivered to clients? This involves maintaining state at the server (using something like a message queue maybe?) and becomes even more complicated when you bring in horizontal scalability of the subscriptions server.

We support live queries because they are easy to reason about, very expressive, extremely performant and the server need not store any state (in our implementation).

heyrict commented 5 years ago

@0x777 Well, I like hasura's performant design and don't think bundling another layer inside to maintain state is a good choice. However, there are still use cases for event based subscriptions and I've seen people using hasura as a remote schema and use custom logic to handle subscriptions. May be it is reasonable to inform users about the possible workarounds in the docs?

bkniffler commented 5 years ago

One thing I've found is, it is very hard to write a mutation that would optimistically update a subscription with apollo, or maybe even impossible, since apollo doesn't seem to have build in mechanisms to automatically update subscriptions (using optimisticResponse) and there is no possibility to read/write subscriptions from/to cache. This must be due to the fact that, as @heyrict states, most implementations and even apollo docs use event based subscriptions instead of live queries.

Would love to see some workaround for live-queries + optimistic response, I couldn't manage to get this combination to work with apollo.

heyrict commented 5 years ago

@bkniffler I am not sure if I have understand live-queries + optimistic response properly. You CAN use live queries with optimistic response.

If you are using

<Subscription subscription={SOME_LIVE_QUERY}>
   {this.renderLiveQuery}
</Subscription>

Then the subsciption data will always reflect what SOME_LIVE_QUERY queries to (though with some latency), and is not designed to be modified.

If you want to use optimistic response to update the live query, you should store query data in local cache with a Query component. In this case, you have to write a custom update function in subscribeToMore as apollo will not automatically merge the live query with the cache.

<Query query={INITIAL_QUERY}>
  {params => <QueryRenderer {...params} />}
</Query>
class QueryRenderer extends Component {
  componentDidMount() {
    this.unsubscribe = this.props.subscribeToMore({
      query: SOME_LIVE_QUERY,
      update: (...) => {/* Logic to update cache here */},
      ...
    });
  }
  ...
}
RodolfoSilva commented 5 years ago

I believe if hasura add support to redis, this feature will be possible.@0x777

mitar commented 4 years ago

What has this with Redis?

barbalex commented 3 years ago

@heyrict One of the biggest challenges/decisions with event based subscriptions is how does the client recover from loss of events? Does the server store what events are delivered to clients? This involves maintaining state at the server (using something like a message queue maybe?) and becomes even more complicated when you bring in horizontal scalability of the subscriptions server.

We support live queries because they are easy to reason about, very expressive, extremely performant and the server need not store any state (in our implementation).

@0x777

The problem with live queries is that they always deliver all the information.

Say I have a table with 10'000 records. One of them changes. So the live query fires. Now my application needs to find out which record changed in order to update it client side. So it has to compare all 10'000 records!

If event based subscriptions existed, the client could:

This seems so obvious I am sure I must have missed something essential. What have I missed?

nolandg commented 3 years ago

I posted what might be a slightly simpler work-around for some scenarios here: https://github.com/apollographql/apollo-client/issues/5267

maxpain commented 2 years ago

Any updates on this?

marcusrohden commented 2 years ago

hi guys, any updates on this? would be amazing to have the ability to receive the initial data + only the changed data in future updates.

pleclech commented 2 years ago

@marcusrohden What you want is probably looking at the streaming API: https://www.youtube.com/watch?v=C1TDDlXPSmI

maxpain commented 9 months ago

Any updates?

vincentjames501 commented 8 months ago

Also wondering if this has been explored anymore!