New is_online() function does not see all users/rooms

[jkruse]

Some rooms never show up in the output from the new API used in is_online(), even when requesting subsequent pages with the "offset" parameter. Possibly these rooms have "Appear on network sites" turned off or use some other setting which makes CB filter them from this service.

Unfortunately that means they will never be recorded now.

[skybarberom]

How do you use the offset parameter? I have trouble recording two streams

[divenxx]

the api get the top list of performers, but it limits the number to the offset , meaning if you want to get the top 500 you set the number to 500 the bigger the number the bigger the list you get the problem if you want to record someone who is not in the top 500 you need to increase the size i personally reverted to the older version of the function, maybe i will create a PR to use based version based on a command line param or something

[skybarberom]

I am not sure if the room was in the top 500, it was on page >10 for sure and a grep on the log file didn't show it. It is maybe a good idea to give the choice so that it's possible to record also far down streams. Regarding the "Appear on network sites" I'm trying to ask if that's the case, but I don't think so. It's probably the position in the list as you said

[jkruse]

Recordurbate doesn't support the offset parameter at all, only the limit parameter (see bot.py line 77).

But the new API used also has an offset parameter. If you copy the URL out of bot.py you can add an offset to retrieve subsequent "pages" of the streamer list. The API itself does not support retrieving more than 500 streamers in one call (any limit above 500 will result in error), or filtering the output in any way, so it's not very suitable for this kind of thing (namely checking if one specific streamer is online).

I have also reverted back to the previous method. Not sure what exactly the problem was with that, but it works better than the new one. I suggest reverting the commit altogether.

[leet86]

I observed that the old API stopped working and also an issue was raised (see issue #69). Cause was due to not return any actual details for any room; online rooms were being reported as offline.

If the old API is working, reverting is definitely an option.

[jkruse]

Old API works fine for me (it errors occasionally, but so does the new one). Might have been a temporary issue at CB.

[leet86]

Actually, there is no requirement to check if a room is online before starting the youtube-dl process. Youtube-dl will attempt to start a recording for an offline room, but it is smart enough to end quickly, gracefully, and with no errors or video file output.

In testing this, recordurbate was still running fine after more than 24 hours without checking for 'is_online()'. I did not experience any issues with starting/stopping lots of youtube-dl processes. Thoughts on this?

I was able to remove the check for "is_online()" completely, and then all streams start instantly when Recordurbate starts.

[leet86]

@jkruse can you help me test this? I can push the changes on a new branch and confirm that it works for others.

[jkruse]

@jkruse can you help me test this? I can push the changes on a new branch and confirm that it works for others.

Yes, sure!

[divenxx]

i confirm i reverted to the old API and it is still working for me

[jkruse]

I could add that when the old API fails (which it does every so often), it's usually with a 502 Bad Gateway from Cloudflare, indicating some temporary issue at CB. For those sticking with the old API, you could safely ignore those errors.

[skybarberom]

i confirm i reverted to the old API and it is still working for me

What version should I download to use the old API?

[divenxx]

i did it by hand, by copying the old code, there is released version just commits

[oliverjrose99]

tbh at this point its probably just quicker and eaiser to call youtube-dl for every streamer and check if the process is still alive after a min.

[divenxx]

lots of people are running that on a tiny pi