Write @live directive

[mattkrick]

Doing an @live on something like teamMembers should point to getState().cashay.result.teamMembers, which will be an array of normalized docs.

[mattkrick]

consider the following:

query {
  getTeamById(id: $id) {
    id
    name
    teamMembers {
      id
      preferredName
      projects {
        id
        content
      }
    }
  }
}

This is beautifully simple as a 1 time query, but impossible as a subscription. To make it live, we could use cashay.computed like we have before, but the subscirbe calls are repetitive, it's not immediately clear what they do, and it is too easy to abuse.

A more elegant solution might be a directive:

const str = `query {
  getTeamById(id: $id) {
    id
    name
    teamMembers @live {
      id
      preferredName
      projects {
        id
        content @live
      }
    }
  }
}`

Proposed API: @live(channel)

• channel: defaults to the name of the field provided (eg teamMembers). Should almost always be blank.

Cashay is opinionated in that it requires a Pub/Sub model for subscritpions. The channel will serve as the channel, but we'll need a way to create a key.

Similar to how resolve works on the server, we could resolve a key, with a default looking like this:

cashay.query(str, {
  live: {
    [channel]: (source, args, context, refs) => source[idFieldName] 
  }
}

How it works: The result is populated from cashay's getField(getState().cashay.result[field]). It then gets denormalized, same as any other. An incoming sub would trigger an invalidation of the denorm result.

The question remains: how to start a subscription? We'd still have to use cashay.subscribe because there is a 1-to-1 relationship between a sub and subscriber. Also, the requested object must include all fields that the sub could ever want, since requiring a field later on means starting a new subscription, which could have side effects. So, we'd still have something like cashay.subscribe(subscriptionString, subscriber). We won't need variables since those are just used to compute a key. The op is just the channel name, which will also be provided for us. The dep, invalidated are obviated because computed will go away.

The subscriber will be a bit different as well:

subscriber(channel, key, handlers) This makes things much cleaner because now there's no need to keep a subscriptions array lookup.

Tangent for invalidation: It might be worthwhile to minimize invalidations at the field level instead of the object level. However, cashay current invalidates at the object level to save space (and the computations required to check those invalidations). It looks something like invalidationCount * bigCost ?= memoryCost + (invalidationCount * littleCost)

For infrequent queries, where invalidationCount is small, the former is better. For subs, where invalidationCount is much, much greater, it's worth it to invalidate by field. For queries that are frequntly polled, everything this the same except the protocol. So, we should allow for the dev to fine tune their invalidation. A reasonable default might be to invalidate on objects for queries, and fields on subs. Or maybe just invalidate by field by default for everything, and let the memory-conscious scale back? If the difference is huge, cashay could even learn which queries are hot & turn those to by-field invalidations.

[mattkrick]

thoughts? @jordanh

[jordanh]

I love everything about this.

When using this for a react container, would you call the cashay.subscribe method from from the component constructor? Then all subsequent cashay.query methods from mapStateToProps?

[mattkrick]

i'm still not sure where to call subscribe from. if i do it from the constructor, then it won't find anything in mapStateToProps, so it'd probably need to be a peer or parent of the connect container. ideas?

[mattkrick]

one consideration-- if it's a topmost (query-level) request, like say team, the query doesn't exist for that, and we'd forward directly to subscriptions. Is it wrong to call cashay.query(subscriptionName)?

[jordanh]

Ah, of course, the constructor won't work in that case. Another component decorator would work...I could see that. Be nice to avoid it if we can.

In your second case:

Is it wrong to call cashay.query(subscriptionName)?

I'm not sure I follow. I need to see a bit more of an example. Your saying you'd always have two query statements in mSTP?

Exploring a bit more, how about another directive?

const str = `query {
  getTeamById(id: $id) @subscribe(subscriptionMutation) {
    id
    name
    teamMembers @live {
      id
      preferredName
      projects {
        id
        content @live
      }
    }
  }
}`

First time through, it sets up the subscription for you. On subsequent calls, the return of the cashay.query call always yields the unsubscribe handler.

subscriber is passed as an option.

Doh, thinking about this more, it doesn't get us any closer because the subscriptionMutation would would need to specify all it's fields....

[jordanh]

Could you not write fatter query? Perhaps that would be more expressive?

  getTeamById(id: $id) {
    id
    name
    teamMembers @live {
      id
      preferredName
      projects {
        id
        content @live
      }
    }
  }
  subscribeTeamMembers @subscription {
    // fields
  }
  subscribeProjectContent @subscription {
    // fields
  }

If @subscription decorator is found then subscribeTeamMembers and subscribeProjectContent are passed the same vars as previous query. Subscriber method is passed in options object:

{
  subscription: {
    subscribeTeamMembers: { subscriber: teamMemberSubscriber },
    subscribeProjectContent: { subscriber: teamMemberSubscriber }
  }
}

Something like that?

[jordanh]

There are two challenges:

1. Subscriptions are not 1:1 with @live decorated fields, therefore a subscription might be used by multiple queries
2. You only want to call the subscription once

[mattkrick]

thinking about this more, we have everything we need to call cashay.subscribe ourselves. we have the channel & the function to resolve the key. The only missing piece is the subscriber, which may vary for each @live. If we treated them like options, it'd look something like this:

const options = {
    live: {
      [channel]: {
        resolve: (source, args) => source[idFieldName],
        subscriber: require('./subscriber')
      }
    }
  }

The goal is to make this as rare as possible. Chances are, the resolve function will always be the default function shown above, so if that's not supplied, we'll never need to see it. The subscriber can change frequently, but we only use 2 (1 for the DB, 1 for Presence). We could give the singleton a default subscriber and only pass in a different one for Presence.

[jordanh]

Yeah, that looks awesome. How will you get the name of the graphql subscription to call? Will that be an option too?

[mattkrick]

cashay don't care. it just gives channel, key, handlers to the subscriber function. channel, key should be enough data for a user to do whatever they want.

[mattkrick]

remaining questions:

• should the array of documents that comprises the sub be persisted in redux, or is persisting in the singleton good enough? A subscription will always return the same shape, so even though a subscriptionString doesn't exist, we can make one based on what we get back. Persisting in redux means a little more work to denormalize the docs, but it also means we can persist it. Being able to persist an array of subs could be useful if we subscribe with a cursor (eg get all documents where updatedAt is greater than the last document we received.
• where should the subscriber be called? At first I put it in the denormalization step, but I think it might be better in the parseAndInitializeQuery.
• Assign dependencies in parseAndInitializeQuery. Since the decorator is static, doing it here makes the most sense.
• What does unsubscribe/resubscribe look like? unsubscribe is no longer on a sub-by-sub basis, rather each query has many subs. Unsubscribing from all subs within a query seems natural, but if there are 2 queries that depend on the same sub, it might be unintuitive to unsub from both when the request was only for 1. For example, Query A relies on Sub 1,2. Query B relies on Sub 2,3. If I unsub from A, I unsubscribe 1,2. Should I also unsub from 3? Probably not, it's up to the user to properly close it out. Unsubscribing might take 2 approaches: Unsub when no component needs the sub anymore, or let the user call unsub. The former is really hard to detect since redux protects the listeners, to do it right, I'd have to use a modified connect that called home each time to say it is still there. Additionally, it seems a little too magical.So, it may be best to let someone call cashay.unsubscribe(op, key. {force: false}) and that triggers an unsub for all the subs within the query. The problem there is we may want to conditionally unsubscribe. For example, only unsub if nothing else is using it (force = false). So, perhaps an unsubscribe just removes that query from the subscriptions dependency, and then if the dependents list is 0, unsub. A resubscribe should occur whenever something that needs it gets mounted.

[mattkrick]

• The array is now normalized & added to the redux store. A small ding on performance, but there are plenty of low hanging fruit here if we need to speed it up, the easiest being to only normalize the doc that changed instead of the whole array.
• subscriber can't be accurately handled in parseAndInitializeQuery because if the @live is nested, the dependencies & subscriptions won't be initiated until the result of the parent comes back. This still works if the top-most query has @live, but won't work for children.
• For children, we'll need to call subscribe & add dependencies whenever a query comes in, or setVariableschanges something, or a mutation changes something, or a new subscription comes in. The lowest common denominator for all of these is the denorm function.

[mattkrick]

unsubscribing from these is the tricky part. For example, if i have a query named getMembers and it does teamMembers(teamId) @live, and then I use setVariables to change that, I should remove getMembers from the list of dependencies & if that list is now 0, then I should unsubscribe. Again, the key is detecting when a sub is no longer needed. My dependencies could take the form of a list where each row has channel, channelKey, queryOp, queryOpKey. If setVariables is called, we can create a new list of all the subscriptions required, diff that with the old list, and for anything that just exists on the old list, remove that row. If there are no more queries that depend on it, then we can unsubscribe automatically.

For plain old unsubscribes, we can remove the dependency anytime the query is invalidated. That way, if the query is never recomputed, we know it is no longer in the view layer. Then, when unsubscribe is called, we can safely know whether this was the only thing that needed it.

[jordanh]

You got a lot of thoughts here. My $0.02:

• Don't worry about backing subs in redux for now – let that come later. Start incremental.
• I agree with you, I think reference counting subs is the way to handle "one query has many subs so what does unsub do?" When the reference count is 0, then unsubscribe unless force is given.

FWIW, so far I am tracking how you're computing the reference count for a sub... it looks great so far!

[mattkrick]

closed with v0.20.0