search

haishinio / haishin

A website to transcribe japanese language livestreams and then translate them into english
2 stars 0 forks source link

Streamlink -> FFMPEG pipe causes memory to just keep increasing #95

Open tomouchuu opened 1 year ago

tomouchuu commented 1 year ago

So using docker stats on the server, if a stream is being captured but not watched. Every 2 minutes of so (whenever the thumbnail worker starts) we see a huge spike in the cpu usage for about 30seconds or so and then it goes back to normal.

I'm wondering if there is a better way to get the thumbnail from the stream since I think the current way involves us loading the entire stream and then taking a frame at a certain point of time. Could there be a way to not do this?

tomouchuu commented 10 months ago

Fixed with change to queues I believe! 57daefb0993dd75e9e3283555cdd15a699942d21

tomouchuu commented 10 months ago

Actually still an issue but we debugged it to not be part of the thumbnail worker. Instead it was due to the piping of streamlink to ffmpeg for the restreamer, if we don't have this pipe then I believe there would be no increasing memory issue