search

leerob / next-music-player

A blazing fast, information dense media player built with Next.js.
https://next-tracks.vercel.app
581 stars 49 forks source link

File streaming #2

Closed eric-burel closed 2 weeks ago

eric-burel commented 2 weeks ago

Hi @leerob, I've been studying file streaming in Next.js and Astro lately so I was definitely interested in your "filename" API route. In this line I see that you copy the file in memory. I suspect this may have a big RAM impact at scale, since the variable stays in memory until it's garbage collected. So I've crafted a file streaming demo using an experimental Node.js feature to convert fs streams in to web streams (the data structure are slightly different and incompatible unless we transform them).

I could observe a positive impact locally using a very big zip file, the stream version has a tiny spike and the copy-in-variable have a longer spike, because the variable stays in memory. In a serverless environment this could bloat functions that have a RAM < 1gb as a default (we even use a lower memory of 768mb for the state of js to save costs).

I'd be eager to get your feedback: would it have a significant impact on your app?

leerob commented 2 weeks ago

I mostly moved away from using that API route to remotely stored audio files in Vercel Blob, mainly because the size of putting the audio files directly in Postgres could be large. I would have needed to refactor how I was doing data fetching, to fetch songs separately likely, but didn't dig into it much further because Blob was an easier choice.

eric-burel commented 2 weeks ago

Right Blob storage makes this a breeze ! and avoids having to care too much about file management in your own app.