search

Stremio / stremio-beamup

🛠️ A platform as a service (PaaS) hosting for Stremio addons: as easy a Heroku, without the restrictions
MIT License
24 stars 7 forks source link

Sporadic issues with beamup server. #57

Closed svetlozardraganov closed 6 months ago

svetlozardraganov commented 7 months ago

Hello,

Please accept my apologies for the unclear title. I'm facing various issues with the beamup-server I don't know how to debug and fix.

I have developed an addon that works fine locally and have been working fine for few weeks after initially being deployed to the beamup server. Recently I have added the addon to the community list and I noticed the addon became very unstable.

The addon is supposed to provide streams for the cinemeta-addon but recently it either doesn't provide any streams or the number of the streams is much smaller than it should be. Sometimes the addon works as expected, everything is working as it is supposed to work and as it works locally.

The addon is not used heavily by other users, the usage is very low and I don't think the issues is related to the fact the addon is now available in the community list.

Sometimes I got the following error: FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory

I checked the memory usage of the addon and I couldn't find any memory leaks. The addon consumes under 50MBs on my local machine. Does this hit any limitations or restrictions?

Now, the most difficult and discouraging part for me - the log files in beamup-server. I can't download the full log from the following page: https://baby-beamup.club/ The download process starts successfully but after short amount of time the download process stops and it timeout.

If I download the logs using beamup.com logs the download is partially completed, but it's not very useful because:

Without access to the server, and without complete and up-to-date log files I don't know how to troubleshoot this issue.

Is there a way to get the full and up-to-date log file? Is there a way to remote existing log file to clean up outdated information? Is it possible the reason is in the beamup-server rather than in my addon? Any advices what else I could do on my end to troubleshoot and hopefully fix that issues?

dexter21767-dev commented 7 months ago

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory definitely means there's a memory leak. i would suggest using something like swagger-stats to check the memory usage of the deployment and how many requests you're getting. and if u want up-to-date logs just run beamup logs and it should give u the latest logs.

Is it possible the reason is in the beamup-server rather than in my addon?

the reason is definitely in the addon. but u shouldn't be discouraged by this. probably your addon is getting thousands or hundreds of requests, as u will be getting a request every time a user opens an episode or a movie from the cinemeta. if u get one request per second and your addon takes longer than a second to process that request u will eventually have a lot of on going requests. that's why u should probably add timeout to close requests rather than keeping them open. and probably some caching can help

but no one can say for sure without seeing the actual code for the addon.

svetlozardraganov commented 7 months ago

Thanks for the provided guidance, I'll check the swagger-stats module, hopefully it will help me to figure it out where the issue comes from.

but no one can say for sure without seeing the actual code for the addon. I can provide access to the code if you would like to check it.

svetlozardraganov commented 7 months ago

I'm noticing an improvement in the performance of my addon since yesterday. I haven't done any changes on my side, is there a chance someone else updated something?

I'm now able to download the full-log files successfully from https://baby-beamup.club/ - this was very useful because now I can see that my addon is used much heavily as I initially though.

The log files are up to date and in right order - before yesterday the order was mixed up and the information was from several hours.

The memory issue is also gone, I checked the entire log and there were no error for 8-hours addon up-time.

This pretty much narrows down the issue to the number of the requests and processing time of the addon. I can see from the logs that the addon often receives 2-3 requests per second, however the processing time of a single request might take more than a second to be completed. The reason for this is the website the addon is getting information doesn't have public API and I need to scrape multiple pages to get the data.

I think this might be improved on stremio-addon-sdk side by implementing a middleware before the defineStreamHandler() and the play-execution. I submitted an issue about this here few months ago.

What can I do on my side to improve the addon. The addon will gets more requests per second that it can process?

svetlozardraganov commented 7 months ago

Hi again,

I'm still monitoring the performance of the addon and I'm noticing the following:

Not sure what's going on but I can't find any issues with my addon. The performance of the server looks very inconsistent and I'm still thinking the issues is on the server side.

dexter21767-dev commented 7 months ago

@svetlozardraganov

Some of the request I'm sending to the addon are not received. I'm logging all incoming request in the log and later when I check the log the request I made from Stremio are not even registered.

that's the desired behavior, as responses are mostly cached (on nginx in the beamup server, on cloudflare, and locally on your machine), that should help reduce the load on your addon. but u can set it to not use cache if u want to (highly don't suggest).

also, other users reported issues with beamup lately especially during high load times. hopefully the issues will be fixed soon.

and yes, u can share the code, i will try to help.

svetlozardraganov commented 7 months ago

Thanks for the details @dexter21767-dev

I'm noticing a very nice improvement in the BeamUp dashboard: image

This would be definitely very helpful.

I've just shared github-repository of my stremio-addon. It would be much appreciated if you can have a look at it and let me know how I can improve it in order to achieve better performance and stability.

svetlozardraganov commented 7 months ago

Unfortunately today I wasn't even able to upload my addon, tried multiple times but to no eval: image

dexter21767-dev commented 6 months ago

@svetlozardraganov that error actually means that the connection between your PC and the server is slow (timing out) but that doesn't mean the deployment stopped or failed.