useObservables for react hooks

[ericvicenti]

Hey all, I've found withObservables to be super handy, but now with react hooks I've been using a simple "useObservables" instead:

import { useState, useEffect } from 'react';

export default function useObservable(observable) {
  const [value, setValue] = useState(observable && observable.value);

  useEffect(
    () => {
      if (!observable || !observable.subscribe) {
        return;
      }
      const subscription = observable.subscribe(setValue);
      return () => subscription.unsubscribe();
    },
    [observable]
  );

  return value;
}

Usage:

function MyView({ observable }) {
  const currentValue = useObservable(observable);
  return <Text>{currentValue}</Text>
}

Note: it is best to use ObservableBehavior so that the observable.value can be accessed synchronously for the first render

[kilbot]

I'm just getting started with WatermelonDB, Observables and React hooks. I had some problems with the above code causing endless re-renders .. I suspect due to the ObservableBehavior as mentioned, but I don't fully understand what that means 😞

However, I did have success with this useObservable library, so I thought I'd just leave it here in case it helps someone else 😄

[kilbot]

The useObservable hook I am currently using is:

import { Subject, Subscription } from 'rxjs';
import { useEffect, useMemo, useState } from 'react';

export default function useObservable(observable, initial, inputs = []) {
  const [state, setState] = useState(initial);
  const subject = useMemo(() => new Subject(), inputs);

  useEffect(() => {
    const subscription = new Subscription();
    subscription.add(subject);
    subscription.add(subject.pipe(() => observable).subscribe(value => setState(value)));
    return () => subscription.unsubscribe();
  }, [subject]);

  return state;
}

Usage:

function MyView({ observable }) {
  const currentValue = useObservable(observable, 'initial', [observable, triggers]);
  return <Text>{currentValue}</Text>
}

I was going to post a PR to this repository ... but import { useObservable } from '@nozbe/with-observables' feels a little weird. Perhaps it would be better to have some sort of monorepo for hooks? Especially if @brunolemos has ideas for more hooks :smile:

[radex]

I was going to post a PR to this repository ... but import { useObservable } from '@nozbe/with-observables' feels a little weird.

I know! I think it's best to post it anyway, and then figure out what's the best name… I imagine withObservables will be obsolete/deprecated in a year when everyone switches to Hooks anyway...

[radex]

@kilbot as for your hook, I think it would be cleaner and more performant to avoid extra Subject, and just set state based on observable subscription. Another issue is the need for initial prop. If you have it - it's great, but it would be best to use Suspense to prevent further render and just wait until we can get our hands on the value subscribed to. WDYT?

[kilbot]

I was having a problem with endless loops, ie: the first subscription was setting the state which triggered a rerender which started the process again. The extra Subject was an effort to get around that. However, I only started learning about observables when I wanted to use WatermelonDB .. so I'm a bit out of my depth 😓

I started rewriting the example app for WatermelonDB with typescript and hooks - mostly as a learning exercise for myself - I'll take another look at it this weekend to see if I reproduce the rerender issue I was having in my app.

[kilbot]

Hi @radex, I've created some example code to illustrate the infinite loop problem I am having. Please compare these two Netlify builds: useObservable with extra Subject and useObservable without. You'll see the subscribe/unsubscribe loop in the console.

Click here to compare the code(updated below) and here is the initial subscription which causes the problems.

It's a bit tricky to share code because I can't load everything into CodeSandbox .. but let me know if you spot anything!

[kilbot]

I've just had a look at this again with fresh eyes and realise it is the removal of the observable as a dependency not the addition of the Subject which stopped the infinite loop, eg:

export default function useObservable(observable: Observable, initial?: any, deps: any[] = []) {
  const [state, setState] = useState(initial);

  useEffect(() => {
    const subscription = observable.subscribe(setState);
    return () => subscription.unsubscribe();
    // eslint-disable-next-line react-hooks/exhaustive-deps
  }, deps);

  return state;
}

This seems to work okay.

It goes against the advice from the React team to remove the observable dependency, so there still may be some issues that I'm unaware of ... I'll have to do some more reading 😅

[ericlewis]

This seems very similar to how react-apollo-hooks works!

[radex]

FYI! I'm working on a fully supported, tested & high-performance solution for hooks :)

This is not easy, because:

• asynchronicity means we might not have the data on first call to useObservable, but caching means we might have
• hooks can't cause the component to bail out early and return null (the way HOCs can)
• Suspense is not fully supported yet
• there's no support for disposing resources for uncommitted renders, which means we have to do some hacking to get the right performance & DX (See: https://github.com/facebook/react/issues/15317#issuecomment-491269433 _

here's some snippets of the work I'm doing.

Basic useEffect-based implementation:

export const neverEmitted = Symbol('observable-never-emitted')

export function useObservableSymbol<T>(observable: Observable<T>): T | typeof neverEmitted {
  const [value, setValue] = useState(neverEmitted)

  useEffect(() => {
    const subscription = observable.subscribe(newValue => {
      setValue(newValue)
    })
    return () => subscription.unsubscribe()
  }, [observable])

  return value
}

WIP highly-optimized implementation, but one that only works on BehaviorSubjects, and cached Observables (essentially, observables that can emit a value synchronously):

export function useObservableSync<T>(observable: Observable<T>): T {
  const forceUpdate = useForceUpdate()

  const value = useRef(neverEmitted)
  const previousObservable = useRef(observable)
  const subscription = useRef(undefined)

  if (observable !== previousObservable.current) {
    throw new Error('Passing different Observable to useObservable hook is not supported (yet)')
  }

  if (subscription.current === undefined) {
    const newSubscription = observable.subscribe(newValue => {
      value.current = newValue

      if (subscription.current !== undefined) {
        forceUpdate()
      }
    })
    subscription.current = newSubscription
  }

  // TODO: GC subscription in case component never gets mounted

  if (value.current === neverEmitted) {
    throw new Error('Observable did not emit an initial value synchronously')
  }

  useEffect(() => {
    return () => subscription.current.unsubscribe()
  }, [])

  return value.current
}

The plan is to take advantage of Suspense (or simulated suspense using error boundaries), so that you can just call const value = useObservable(query), without having to worry about initial value, and it does the right thing.

But you'd still have to wrap it somewhere with a <Supense> or a custom <ObservableSuspense> or something like that. I don't think there's a way around it.

I'd also like to publish an official prefetched component, so that you can do:

<Prefetch observables=[query, otherQuery, etc]>
  <SomeComponentTreeThatUsesUseObservableHook />
</Prefetch>

this would be much faster than relying on suspense, since you can start db operations early, and by the time you get to rendering components, you have a synchronous data source, instead of constantly rolling back partial renders.

Would someone be willing to help out with writing tests for this?

[kilbot]

This is great @radex! I would be keen to help out in any way I can.

I'll leave a link to mobx-react-lite here, just in case it is useful. There is a discussion on getting MobX to work with react hooks in concurrent mode, I guess there may be some parallels for WatermelonDB?

[radex]

@kilbot Thanks! I've seen that thread and mobx-react-lite, and I was planning to spend some time next Friday digging deeper into it to understand how they did it ;) Although MobX doesn't really have asynchronicity issues AFAICT.

[danielkcz]

Just want to point out we have mobx-react-lite@next which includes experimental support for Concurrent mode. Feel free to try that.

[radex]

@FredyC Thanks! I'll check it out!

[crimx]

observable-hooks supports Observables and Suspense which I think is super handy to use in conjunction with withObservables.

[radex]

@crimx Amazing work! Looks like roughly what I was going for, but never had time to implement (yet) :) … although I do see some potential perf issues. I will check it out in the coming months...

[crimx]

That would be great! Much appreciated🍻.

[gliesche]

Hi there. Any news on this?

[radex]

@gliesche Not much! I recently got back to looking at it, and implemented an useModel() hook (not ready to be open sourced yet), but as you can see from the whole discussion, making a generic useObservables without a HOC, without compromises, and without forcing synchronous Watermelon is… well, very complicated :)

[gliesche]

@gliesche Not much! I recently got back to looking at it, and implemented an useModel() hook (not ready to be open sourced yet), but as you can see from the whole discussion, making a generic useObservables without a HOC, without compromises, and without forcing synchronous Watermelon is… well, very complicated :)

Thanks for the update. I tried observable-hooks, but that didn't really work. I'd really be interested in any working solution that uses hooks and observes WatermelonDB collections / models.

[radex]

@gliesche For context: what problems are you looking to solve with a hook-based WatermelonDB observation? Do you just care about nice, consistent API? Or are there composition/performance/other problems you're running into because of this?

[gliesche]

@gliesche For context: what problems are you looking to solve with a hook-based WatermelonDB observation? Do you just care about nice, consistent API? Or are there composition/performance/other problems you're running into because of this?

Consistent API is one main point, yes. Though I am not sure, if hook based observation is needed for custom hooks...

[crimx]

FYI since the last conversation observable-hooks had been refactored a little bit and had been made concurrent mode safe.

@gliesche what problems did you encounter? AFAIK watermelon observables are just plain RxJS observables so they should just work out of the box right?

[radex]

(@crimx FWIW we're moving away from RxJS internally for performance reasons, but yes, the outside API is RxJS-compatible)

[gliesche]

FYI since the last conversation observable-hooks had been refactored a little bit and had been made concurrent mode safe.

@gliesche what problems did you encounter? AFAIK watermelon observables are just plain RxJS observables so they should just work out of the box right?

@crimx I guess I don't get the API right, I tried something like:

  const database = useDatabase();
  const input$ = database.collections.get('posts').query().observe();
  const [posts, onPosts] = useObservableState(input$, null);

Which results in a TypeError: rxjs_1.isObservable is not a function

[crimx]

isObservable is included in RxJS. What RxJS version are you using?

This should work

const database = useDatabase();
// reduce recomputation
const input$ = useObservable(() => database.collections.get('posts').query().observe());
// if the first parameter is an Observable `useObservableState` returns emitted values directly.
const posts = useObservableState(input$, null);

@radex is the observable from watermelon hot and with inital value? I am trying to implement a hook that subscribe synchronously. In concurrent mode it is only safe if the observable is hot.

[gliesche]

isObservable is included in RxJS. What RxJS version are you using?

The version of the metro bundler is being used: rxjs@5.5.12

[crimx]

@gliesche observable-hooks is meant to work with RxJS 6. Is there a reason that you still use RxJS 5? Is it a legacy project?

[gliesche]

@gliesche observable-hooks is meant to work with RxJS 6. Is there a reason that you still use RxJS 5? Is it a legacy project?

It is not a legacy project. It's a react native project v0.62.2. Do I need to install the peer deps manually? I did not explicitly add a version of rjxs.

[crimx]

Do I need to install the peer deps manually?

Yes. Read this article if you are interested.

What we need is a way of expressing these "dependencies" between plugins and their host package. Some way of saying, "I only work when plugged in to version 1.2.x of my host package, so if you install me, be sure that it's alongside a compatible host." We call this relationship a peer dependency.

[radex]

@radex is the observable from watermelon hot and with inital value? I am trying to implement a hook that subscribe synchronously. In concurrent mode it is only safe if the observable is hot.

@crimx It's not guaranteed to be hot — that's a problem. TL;DR: I'm working towards it.

I'm developing Nozbe Teams (the project that birthed WatermelonDB) with the intention on making all DB work synchronous so that React components can render in one shot… This works 100% of the time on web, but not yet on native (where synchronous native modules can't be guaranteed to work all the time - YET).

The longer-term plan is to go back to more multi-threading, but such that you can synchronously get values you need for first render, but can prefetch and cache queries beforehand on separate thread, and have them hot when you render.

[crimx]

Added useObservableEagerState for safely getting synchronous values from hot or pure observables without triggering extra initial re-rendering.

[quolpr]

@radex thank you for your hooks! I tried to use them this way:

  const input$ = useObservable(
    () =>
      database.collections
        .get<NoteModel>(HarikaNotesTableName.NOTES)
        .query(Q.where('id', id))
        .observe(),
    [id]
  );
  const note = useObservableEagerState(input$)?.[0];

But note is still the same even if id changes. Do you have any ideas about what could be the reason? 🤔

[crimx]

@quolpr I think you should use useMemo instead of useObservable. useObservable does not work like that.

[quolpr]

It seems I was wrong from the start. That way works correctly:

  const input$ = useObservable(
    (inputs$) =>
      inputs$.pipe(
        switchMap((val) =>
          database.collections
            .get<NoteModel>(HarikaNotesTableName.NOTES)
            .query(Q.where('id', val[0]))
            .observe()
        )
      ),
    [id]
  );
  const note = useObservableEagerState(input$)?.[0];

The power of RxJS 😛️

[quolpr]

I have another one problem. This code:

const input$ = useObservable(() => note.observe());
const obervedNote = useObservableEagerState(input$);

It doesn't reflect changes(if the field is just changed). Do you have ideas about why? 🤔

[quolpr]

@crimx it seems here

https://github.com/crimx/observable-hooks/blob/master/packages/observable-hooks/src/use-observable-eager-state.ts#L65

forceUpdate() is required, cause the model will have the same reference(cause setState will not trigger a rerender if the value is the same) again and again, even if the values inside are changed. How could we do it and keep the old behavior? We can introduce an option that will change behavior. What do you think?

[crimx]

Na I think it works as expected. It's called useObservableEagerState so it should be just like React setState.

You should instead pipe the observable for needed values. Or useSubscription and handle it manually.

[quolpr]

@crimx then, what do you think about introducing another hook, which would work like useObservableEagerState but will force rerendering? I mean, your library is great, and I want to use useObservableEagerState cause it meets the requirements, but a small thing stops me from using it 😞️

Btw, here is the changes required https://github.com/quolpr/observable-hooks/commit/7b55ad24692502d981bb003057a5658230d2d379

[crimx]

Hey maybe you can make it a set of hooks that is built on top of observable-hooks and for WatermelonDB specifically? (Like useTable)

I think this is anti-pattern for general cases.

[quolpr]

Good idea!

[MoOx]

I am looking for a local database for React Native and ended up here when looking at watermelondb.

After a quick look it seems there is 2 candidates to replace this:

• https://github.com/crimx/observable-hooks
• https://github.com/LeetCode-OpenSource/rxjs-hooks

Are this packages a good replacement? What would you recommend? Since this issue is open since 2018, maybe watermelondb docs could point to one of them?

[crimx]

FYI: If you plan to use RxJS 7, observable-hooks has added support and testing for RxJS 7 recently.

[radex]

Yes, we'll upgrade to RxJS 7 soon.

Having said that, so far I haven't seen and tested a generic observable hook that I'd be willing to endorse. It's not too difficult to get something that "works", but I'll only recommend to switch away from withObservables HOC once a hook solution can be shown to be as performant or more, and actually correct (there's a lot of weird edge cases to take care of).

I know it's frustrating. I'd like to move to hook-only solution myself too. This year, I hope.

[alexyoungs]

Hi @radex, following this thread with interest. Wondered if there was any further movement on it?

[paulrostorp]

I've got a hack to enable re-rendering while using observable-hooks:

import { useObservable, useObservableEagerState } from "observable-hooks";
import { getProjectCollection } from "@app/models/collections";
import { ProjectModel } from "@app/models/Project/ProjectModel";
import { map } from "rxjs";

export const useProject = (id: string): ProjectModel => {
  const input$ = useObservable(
    () =>
      getProjectCollection()
        .findAndObserve(id)
        .pipe(map(project => ({ value: project }))),
    [id]
  );

  const x = useObservableEagerState(input$);
  return x.value;
};

Quite simply, transforming the payload to be wrapped inside a new object on each event forces a new value in the state each time, thereby causing a re-render

[mfbx9da4]

Yes, we'll upgrade to RxJS 7 soon.

Having said that, so far I haven't seen and tested a generic observable hook that I'd be willing to endorse. It's not too difficult to get something that "works", but I'll only recommend to switch away from withObservables HOC once a hook solution can be shown to be as performant or more, and actually correct (there's a lot of weird edge cases to take care of).

I know it's frustrating. I'd like to move to hook-only solution myself too. This year, I hope.

@radex Would you be able to detail

• What are the key metrics you are evaluating for measuring performance of a hooks solution?
• What are the weird edge cases you mention? Do you have a set of complementary HOC tests which could be used to verify a hooks implementation?

A HOC approach is quite outdated now and I believe will be a deal breaker for many users considering 🍉.

[bard]

Usually, lack of a hook API is a deal breaker to me, too, due (to answer @radex's question above) partly to the mindset switch required when working with HOCs in an otherwise full-hook codebase, and partly to the typing issues that often accompany HOCs (of which there seem to be some: https://github.com/Nozbe/withObservables/issues/123, https://github.com/Nozbe/withObservables/issues/102).

However, costs are to be evaluated against benefits, and WatermelonDB is, to the best of my knowledge, one of only two open source libraries that allow offline-first applications without requiring a specific backend (the other being RxDB), which is a hell of a benefit, so I'd still go with the HOC-based API if there were signs of a hook API coming at some point. I couldn't find anything other than this thread, though. I was wondering if it's still on the radar?

[832bb9]

@gliesche For context: what problems are you looking to solve with a hook-based WatermelonDB observation? Do you just care about nice, consistent API? Or are there composition/performance/other problems you're running into because of this?

Just started to adopt WatermelonDB in my RN App. I understand that we need to host observables logic on components level (which unlocks conditional rendering), but what I don't like is that HOC enforces me to split some UI nodes into separate component just to allow it's wrapping. So instead i used withObservables to create ObservablesContainer with children function exposing observed props.

  <ObservablesContainer deps={[]} getObservables={() => ({ some: interval$(1000) })}>
    {(props) => <Text>{JSON.stringify(props)}</Text>}
  </ObservablesContainer>

So now getObservables and children function can accept any other data from scope including useDatabase result.

@radex based on your WatermelonDB API design experience do you have immediate concerns on such approach?

[17Amir17]

Any update?

[okalil]

Hey! This issue has been here for a while... I am still interested on using react hooks wih watermelondb, so I just would like to share an other approach for local first data https://riffle.systems/essays/prelude/ . It's called Riffle, it is not publicly available yet, but it would make possible a super fast reactive synchronous data hook, that can maybe serve as inspiration for new ideas in this project.