supabase / realtime-js

An isomorphic Javascript client for Supabase Realtime server.
https://supabase.com
MIT License
313 stars 54 forks source link

Access token not refreshed for realtime channels after being offline or in standby #274

Open colin-chadwick opened 8 months ago

colin-chadwick commented 8 months ago

Bug report

When a user is offline or in standby mode, the access_token is not refreshed as it requires a POST request to auth/v1/token?grant_type=refresh_token (which obviously fails without a connection). When the user comes back online or is active again, Supabase doesn't automatically refetch the access_token and supply it to realtime channels. Instead, the realtime channels error and show the following message, because they try to connect with the old token:

Bildschirmfoto 2024-01-10 um 21 52 05

Even if the access_token is refreshed afterwards, the channels don't seem to pick up the change. The only way to get the channels to work again is by removing and re-initializing them.

To Reproduce

  1. Set the JWT expiry time of the access_token (e. g. 60 = 1 minute) to a low value, to be able to better reproduce the case where a access_token needs a refresh.
  2. Subscribe to a realtime channel.
  3. Turn off WIFI or go into standby mode.
  4. Wait for the current access_token to expire (in this case after 1 minute).
  5. Turn the WIFI on again or disable standby.

The realtime channel tries to reconnect with the old token (because Supabase wasn't able to refetch it without a connection) and errors out. It doesn't pick up any new tokens once Supabase refreshes them again in the usual interval.

Expected behavior

If any realtime channels are still active after re-enabling WIFI or disabling standby, Supabase should refetch the access_token right away and supply it to the channels. This way, they won't error out.

System information

Additional context

I've built a temporary workaround to the fix the issue, maybe it can help:


import { RealtimeChannel, SupabaseClient } from '@supabase/supabase-js'

export const realtime = (client: SupabaseClient) => {
  let channel: RealtimeChannel | null = null
  let subscribed = false
  let reconnect = true
  let reconnecting = false

  const _subscribe = ({ table }) => {

    if (channel) client.removeChannel(channel)

    reconnecting = true

    return (
      callback: (payload: any) => void,
    ) => {
      channel = client
        .channel('realtime')
        .on(
          'postgres_changes',
          {
            event: '*',
            schema: 'public',
            table,
          },
          (payload) =>  {
            callback(payload)
          },
        )
        .subscribe()
        // @ts-expect-error
        .on('phx_reply', {}, async (payload) => {
          if (
            payload.status === 'error' &&
            payload.response.reason.includes('Invalid token')
          ) {
            await client.removeChannel(channel)

            await client.auth.refreshSession()

            subscribed = false

            reconnecting = false
          }
        })
        // @ts-expect-error
        .on('system', {}, async (payload) => {
          if (
            payload.extension === 'postgres_changes' &&
            payload.status === 'ok'
          ) {
            subscribed = true

            reconnecting = false

          }
        })

    }
  }

  return {
    from: (table: string) => {
      return {
        subscribe: (
          callback: (payload: any) => void,
        ) => {
          let timer: NodeJS.Timeout

          const reconnectSubscription = () => {
            clearTimeout(timer)

            timer = setTimeout(() => {
              reconnectSubscription()

              if (
                reconnect &&
                !subscribed &&
                !reconnecting &&
                document.visibilityState === 'visible'
              ) {
                return _subscribe({ table })(callback)
              }
            }, 1000)
          }

          reconnectSubscription()

          return _subscribe({ table })(callback)
        },
        unsubscribe: async () => {
          if (!channel) return

          await client.removeChannel(channel)
          channel = null
          reconnect = false
        },
      }
    },
  }
}
ghost commented 7 months ago

I can confirm that I'm experiencing this also.

colin-chadwick commented 7 months ago

Could you look into this? 👋 @w3b6x9

ghost commented 7 months ago

Is anyone else experiencing obscenely long cookies (such as using Keycloak as an auth provider)? I am noticing 2 or more cookies for a single login, causing a ton of errors and seemingly this one too. I documented it here https://github.com/supabase/supabase-js/issues/963 but does anyone else experiencing this issue specifically (realtime refresh failing) also have multiple auth cookies?

erdemdev commented 6 months ago

Thıs needs to be resolved asap.

create-signal commented 3 days ago

This is still broken. Here's a simpler version of the temporary workaround that seems to work ok.

Essentially, what is does is watch the channel for a "Access token has expired" response, sends a "auth.refreshSession()" request, then discards the open channel and creates a new one.

I've created a module - e.g. supabase-realtime-bug.ts

import { RealtimeChannel, SupabaseClient } from '@supabase/supabase-js'

// Shares the refresh promise between all channels
let refreshPromise: Promise<void> | undefined

export function fixSupabaseRealtimeBug(
  supabase: SupabaseClient,
  newChannel: () => RealtimeChannel,
  onNewChannel?: (channel: RealtimeChannel) => void,
) {
  const currentChannel = newChannel()

  currentChannel
    //@ts-expect-error
    .on('system', {}, async ev => {
      if (ev.status == 'error' && ev.message.includes('Access token has expired')) {
        await currentChannel.unsubscribe()
        await supabase.removeChannel(currentChannel)
        if (!refreshPromise) {
          refreshPromise = supabase.auth.refreshSession().then(() => (refreshPromise = undefined))
        }
        await refreshPromise
        fixSupabaseRealtimeBug(supabase, newChannel)
      }
    })

  onNewChannel?.(currentChannel)
}

and then I can reuse it

let channel: RealtimeChannel | undefined = undefined

fixSupabaseRealtimeBug(
  supabase,
  () => {
    return supabase
      .channel('db-changes-test')
      .on(
        'postgres_changes',
        {
          event: '*',
          schema: 'public',
          table: 'days',
        },
        payload => console.log(payload),
      )
      .subscribe()
  },
  // Use this callback if you need to keep a reference to the current channel
  newChannel => {
    channel = newChannel
  },
)

Looking forward to a fix for this!

filipecabaco commented 3 days ago

@create-signal are you seeing this with supabase-js? if so which version?

I also think there's some element on supabase-js where we're not properly setting the token back and at first we thought it could be a supabase-js version specifically

create-signal commented 3 days ago

@create-signal are you seeing this with supabase-js? if so which version?

I also think there's some element on supabase-js where we're not properly setting the token back and at first we thought it could be a supabase-js version specifically

I'm testing with supabase-js version 2.45.4

The issue occurs when the auth module stops refreshing the access token because the device is asleep or idle. When the device resumes the asynchronous "refreshSession" method runs and the socket sends an "access_token" message simultaneously (with the old access token). So the logs will look like this:

push realtime:db-changes-test access_token (387) {access_token: 'OLD_TOKEN'}

receive error realtime:db-changes-test system  {message: 'Access token has expired: [message: "Invalid token", claim: "exp", claim_val: 1726823673]', status: 'error', extension: 'system', channel: 'db-changes-test'}

receive  realtime:db-changes-test phx_close (1) {}

channel close realtime:db-changes-test 1 undefined

AUTH EVENT -> TOKEN_REFRESHED (the event that contains the new token)

An easy way to test this without putting your computer to sleep is to

filipecabaco commented 3 days ago

perfect!!! thank you for reporting this and a potential fix 🙏 I will check with the remainder of the team as this can also be the source of another issue we're seeing where it seems realtime just tries to connect with expired tokens.

create-signal commented 3 days ago

Just FYI, i think the push realtime:db-changes-test access_token (387) {access_token: 'OLD_TOKEN'} message in that log was a red herring and is completely unrelated

It looks like the backend server sends an 'error' and a 'phx_close' event for each open channel when the access token expires, which then triggers RealtimeChannel to remove itself from RealtimeClient.channels[] (L175 of RealtimeChannel.ts)

Using RealtimeClient.setAuth() (like supabase-js does) after the channel is removed from that array won't have any effect, when ideally it would be able to assign the new access token to the channel and "rejoin()"