haykkh / what-the-playlist

what playlist was that song in again??
https://whattheplaylist.com
6 stars 2 forks source link

feat: fetch tracks of all playlists #38

Closed haykkh closed 2 years ago

haykkh commented 2 years ago

First, let's modify plugins/api.ts' spottyFetch and spottyPagedFetch to work a little better and take params

  1. let spottyFetch take params as options

    const spottyFetch = async <returnDataType>(uri: string, { params } = { params: {} }) =>
    await useFetch<returnDataType>(`${config.spotifyApiUrl}${uri}`, {
    headers: { Authorization: `Bearer ${authStore.token.access_token}` },
    params
    })
  2. make spottyFetch check the uri to see if it's a full link or just the stuff after config.spotifyApiUrl

    const spottyFetch = async <returnDataType>(uri: string, { params } = { params: {} }) =>
    await useFetch<returnDataType>(uri.substring(0, 4) === "http" ? uri : `${config.spotifyApiUrl}${uri}`, {
    headers: { Authorization: `Bearer ${authStore.token.access_token}` },
    params
    })
  3. make spottyPagedFetch take params and reuse spottyFetch instead of useFetch in the while loop

    const spottyPagedFetch = async <returnDataType extends object>(uri: string, { params } = { params: {} }): Promise<returnDataType[]> => {
    try {
    interface ISpotifyPagesResponseWithReturn extends ISpotifyPagesResponse {
      items: returnDataType[]
    }
    
    const { data } = await spottyFetch<ISpotifyPagesResponseWithReturn>(uri, { params })
    
    const allData = data.value.items
    
    while (data.value.next) {
      const { data: newData } = await spottyFetch<ISpotifyPagesResponseWithReturn>(
        data.value.next,
        { params }
      )
    
      newData.value.items.forEach(item => allData.unshift(item))
    
      data.value = newData.value
    }
    
    return allData
    } catch (error) {
    // eslint-disable-next-line no-console
    console.log(error)
    }
    }
haykkh commented 2 years ago

Unsurprisingly i'm getting rate limited by the API (sometimes) when I try to fetch the tracks concurrently.

// stores/music.ts

async fetchPlaylistsSongs (): Promise<IPlaylist[]> {
  if (!(this.playlists.length > 0)) {
    await this.fetchPlaylists()
  }

  this.$patch(async (state) => {
    state.playlists = await Promise.all(this.playlists.map(async (playlist: IPlaylist) => ({
      ...playlist,
      tracks: await this.$nuxt.$spottyPagedFetch(`/playlists/${playlist.id}/tracks`, {
        params: {
          fields: "next,items(track(name,album(name)))"
        }
      })
    })))
  })

  return this.playlists
}

If i loop over the playlists sequentially all is well (except it takes 5x as long)

// stores/music.ts

async fetchPlaylistsSongs (): Promise<IPlaylist[]> {
  if (!(this.playlists.length > 0)) {
    await this.fetchPlaylists()
  }

  const bb = []
  for (let i = 0; i < this.playlists.length; i++) {
    bb.push({
      ...this.playlists[i],
      tracks: await this.$nuxt.$spottyPagedFetch(`/playlists/${this.playlists[i].id}/tracks`, {
        params: {
          fields: "next,items(track(name,album(name)))"
        }
      })
    })
  }

  this.$patch((state) => {
    state.playlists = bb
  })

  return this.playlists
}
haykkh commented 2 years ago

Had a lot of trouble with nuxt's useFetch and ohmyfetch. Mostly because i accidentally retry(ed)-after X milliseconds instead of X seconds (spotify api "Retry-After" returns a value in seconds, setTimeout uses ms, i was passing the s instead of doing s * 1000.

As it is I have two options I think on how to handle this.

  1. with ohmyfetch's interceptors:
    
    // plugins/api.ts

const wait = (ms: number) => new Promise(resolve => setTimeout(resolve, ms))

const spottyFetch = async (uri: string, { params } = { params: {} }) => await useFetch(uri.substring(0, 4) === "http" ? uri : ${config.spotifyApiUrl}${uri}, { headers: { Authorization: Bearer ${authStore.token.access_token} }, params, async onResponseError ({ request, response, options }) { if (response.status === 429 && response.headers.has("Retry-After")) { await wait(+response.headers.get("Retry-After") * 1000 + 1000) return await useFetch(request, options) } } })


2. capturing the error in `useFetch`
```js
// plugins/api.ts

import { FetchError } from "ohmyfetch"

const wait = (ms: number) => new Promise(resolve => setTimeout(resolve, ms))

const spottyFetch = async <returnDataType>(uri: string, { params } = { params: {} }) => {
  const { data, pending, error, refresh } = await useFetch<returnDataType, FetchError>(uri.substring(0, 4) === "http" ? uri : `${config.spotifyApiUrl}${uri}`, {
    headers: { Authorization: `Bearer ${authStore.token.access_token}` },
    params
  })
  if (error.value instanceof FetchError && error.value.response.status === 429 && error.value.response.headers.has("Retry-After")) {
    await wait(+error.value.response.headers.get("Retry-After"))
    await refresh()
    return { data, pending, error, refresh }
  } else {
    return { data, pending, error, refresh }
  }
}

(extra) i also tried a recursive call to spottyFetch but it just refused to retry the func after the wait. a shame because this solved the type errors that I'm now getting with 1)

// plugins/api.ts

const wait = (ms: number) => new Promise(resolve => setTimeout(resolve, ms))

const spottyFetch = async <returnDataType>(uri: string, { params } = { params: {} }) =>
  await useFetch<returnDataType>(uri.substring(0, 4) === "http" ? uri : `${config.spotifyApiUrl}${uri}`, {
    headers: { Authorization: `Bearer ${authStore.token.access_token}` },
    params,
    async onResponseError ({ response }) {
      if (response.status === 429 && response.headers.has("Retry-After")) {
        await wait(+response.headers.get("Retry-After") * 1000 + 1000)
        return await spottyFetch<returnDataType>(uri, { params })
      }
    }
  })

I prefer 1) the most, it feels the cleanest. But i'm getting a few type errors

Screen Shot 2022-09-06 at 22 55 20

as well as a bunch of them in spottyPagedFetch

Screen Shot 2022-09-06 at 22 55 42
haykkh commented 2 years ago

So i'm having trouble after updating to RC9 and using composables instead of plugins. It seems like it calls useSpottyFetch correctly the first time (in most of my use cases this is when fetching the user at /me, then all subsequent calls just return the result of the first fetch. From my understanding this happens because it's using the same key for useFetch for both endpoints. See here. I can fix this by specifying the key option for useFetch (eg key: uri) but still seems like an issue on nuxt's end

Maybe we shouldn't be using useFetch here at all?

haykkh commented 2 years ago

So by replacing useFetch with $fetch in the composable it all seems to work fine. See if these same issues happened before swapping from plugins --> composable, in which case figure out if we should be using useFetch in composables, or if we should be using composables at all (maybe plugins are better)

haykkh commented 2 years ago

https://github.com/nuxt/nuxt.js/issues/14736

haykkh commented 2 years ago

For now i am using key: uri and then key: generateRandomString() in the onResponseError call.

Follow https://github.com/nuxt/nuxt.js/issues/13924, https://github.com/nuxt/nuxt.js/issues/14303, and https://github.com/nuxt/nuxt.js/issues/14736 for future fixes