search

kii-chan-reloaded / RedTwiBot

A Reddit/Twitch bot that will update your subreddit statusbar when streamers are playing your sub's game
GNU General Public License v3.0
0 stars 0 forks source link

[Feature Request] YouTube Stream Support #2

Closed EthanC closed 6 years ago

EthanC commented 7 years ago

We've recently adapted this Bot for all of our subreddits after using the depreciated Twitchy for so long, so thank you to WolfgangAxel for supporting this project.

I'd like to bring up the idea of YouTube Support for Livestreams, if that's possible sometime in the future for this bot that'd be amazing!

YouTube API: https://developers.google.com/youtube/v3/

kii-chan-reloaded commented 7 years ago

The YouTube API (and most of Google's API's) are a huge pain in the ass to deal with, in my experience. I'll look into this in the future, but this won't be implemented any time soon. The earliest I'd be able to work on this is next week, but I make no guarantees as I have other priorities to attend to.

EthanC commented 7 years ago

Hey, I was curious if you had any plans soon to get this put in?

Sorry to bother, still using this bot, serving us perfectly, just missing YouTube support!

kii-chan-reloaded commented 7 years ago

Hey sorry, this got put on hold and I had forgotten to look into it again.

I've been looking though the API for YouTube for the past few hours, and there's no way to retrieve the status of a livestream that you don't own. This seems incredibly wrong to me, because obviously there's some way that YouTube and apps can do it, but there's literally no documentation on it, outside of the statuses of your own streams. The best I can think to do is make a generic request and download the full html page for each YouTube channel, but even then, I'm not seeing a definitive "live/not live" tag in the html files I tried.

Unfortunately, I don't think I'll be able to implement this as of right now.

EthanC commented 7 years ago

Hey,

Would this be of any help? Seems to be what you're looking for :) http://stackoverflow.com/questions/32454238/how-to-check-if-youtube-channel-is-streaming-live

kii-chan-reloaded commented 7 years ago

I'll see what I can do with it. Looks like it'll work

kii-chan-reloaded commented 7 years ago

Okay, I may have gotten a little hasty with my pushes, but I think this latest revision is finally good to go. Post any bug reports here, and I'll get to them as soon as I can. You'll have to regenerate the configuration.ini file- I wrote my suggestion for how to do that in the changelog.

EthanC commented 7 years ago

Wow, awesomely fast work! It's currently 3AM so I don't have anyone online on YouTube that I can test it on, but I can do it about 12 hours from now. I'll be sure to keep you updated.

There's one little issue I've forgotten to mention, unrelated to YouTube Streaming, I'll open a New Issue for it.

Thank you so much, very appreciated.

EthanC commented 7 years ago

Getting the following error after attempting to start the latest commit.

  File "C:\Users\SERV\Desktop\SidebarStreamers\SidebarStreamers.py", line 341
    if not message.author.name in mods or message.subject in ["username mention","comment reply"]::
                                                                                                  ^
SyntaxError: invalid syntax

Only change I made was renaming configuration.ini to configuration.old.ini, to generate a new one, as recommended.

kii-chan-reloaded commented 7 years ago

Fixed.

EthanC commented 7 years ago

Thanks, can confirm it was fixed.

EthanC commented 7 years ago

Okay, confused myself a bit, thought I opened a new issue for that last error. Still got an issue with YouTube Support.

Couple issues...

First, When generating a new configuration.ini via the Script, YouTube Stream values are placed in the [G] (games) section. However, manually moving them from [G] to [YS] works as a temporary fix.

Second, Received this error while attempting to run the main loop, with the YouTube Streams added to configuration.ini. Also note that the ChannelID is shown instead of the set human-readable name for the YouTube Stream, and the Stream Title is used instead of Stream Game.

UCovvVRCqBocOUiQ6LlB_T_w is streaming Test for reddit
Status of UC2GC7O0nULuhHaeItbNPldw exceeded too many failed attempts. If problems persist with this streamer, open an issue here: https://github.com/WolfgangAxel/RedTwiBot/issues/new
Request response was:
Error!

  Line 19 -> 'charmap' codec can't encode character '\U0001f52e' in position 73: character maps to <undefined>

Retrying in one minute.

Here is a copy of my configuration.ini.

kii-chan-reloaded commented 7 years ago

The streamers adding to the wrong list is fixed. You can keep your configuration file as-is, no need to regenerate it.

As for the channel ID/stream name thing, that's working as expected. I did switch the output to use the readable name you provide instead of the ID, but the bot is set up to print the title for any stream it finds, since this output happens before it checks to make sure they're playing an acceptable game. It will use the channel's title as seen on youtube plus the full stream title for the link in the sidebar if it finds that they are streaming an acceptable game.

This error portion is throwing me for a loop, though. It printing the message for too many attempts looks like something weird happened with the response from youtube (it failed at line 310). When it tried to print the JSON response (line 321), it threw another error about the encoding, causing the whole loop to break. I'm not able to recreate it right now, so I'm not really sure what caused it or what to do about it. The only thing I can think of right now is that maybe their stream title had some strange characters (emoji?) in the title that caused the JSON to read improperly. If that were the case though, I would think it would have failed when translating the JSON into a dictionary at line 295. I've changed how the errors are displayed slightly- I have no clue whatsoever if it will help or not. It's a shot in the dark since that stream isn't active anymore, so I can't recreate what happened.

EthanC commented 7 years ago

Interesting, I ran it again while a different YouTube Stream was live, no errors, however it's not updating the sidebar with the YouTube Stream.

Loading configuration from file...
Configuration file read. Checking for usable values.
Found sections. Checking for values...
Found values for section R
Found values for section T
Found values for section Y
Found values for section M
Found values for section G
Found values for section TS
Found values for section YS
Configuration seems usable. Using C:\Users\SERV\Desktop\SidebarStreamers\configuration.ini
Bot successfully loaded. Entering main loop.
Summit1G is playing League of Legends
Chucky is streaming "ZOMBIES CHRONICLES" SHANGRI LA REMASTERED GAMEPLAY (Black Ops 3 Zombies DLC 5)
05/11/17, 22:47-
    Sidebar was updated
Summit1G is playing League of Legends
Chucky is streaming "ZOMBIES CHRONICLES" SHANGRI LA REMASTERED GAMEPLAY (Black Ops 3 Zombies DLC 5)
Summit1G is playing League of Legends
Chucky is streaming "ZOMBIES CHRONICLES" SHANGRI LA REMASTERED GAMEPLAY (Black Ops 3 Zombies DLC 5)
Summit1G is playing League of Legends
Chucky is streaming "ZOMBIES CHRONICLES" SHANGRI LA REMASTERED GAMEPLAY (Black Ops 3 Zombies DLC 5)

Everything goes as it should, however the sidebar says No active streams.

Same configuration.ini file as above. It's worth noting that only a YouTube Stream is live, as the Twitch Stream is not a game of ours. Could it be that it's just not updating the sidebar with YouTube streams? I can wait until the morning and see if it'll update with Twitch streams if necessary.

kii-chan-reloaded commented 7 years ago

The problem here arises with how YouTube handles streaming and how the streamers handle titling their streams. Twitch has a value for the name of the game someone is streaming, so using full game titles in the configuration file gives desired results. YouTube does not do this. The only thing that YouTube gives to specify a game is the stream title. It's obvious to us that black ops 3 is a game that you'd want the bot to catch, but the bot is technically looking for all of call of duty black ops iii, as you have set up in your configuration. You would have to add black ops 3 = Good to your games list for the bot to pick up that stream. This would go for every nickname any of those games has.

If you add nicknames to the list manually, just remember to use all lowercase letters and ignore = or : in the nickname.

EthanC commented 7 years ago

Understandable, went ahead and added the correct games so that it'll pick up what I'm looking for.

I'm now receiving the following error:

Configuration seems usable. Using C:\Users\SERV\Desktop\sidebarstreamers\configuration.ini
Bot successfully loaded. Entering main loop.
Swiftor is playing Call of Duty: Black Ops III
Summit1G is playing Quake Champions
NoahJ456 is streaming KINO DER TOTEN "PACK A PUNCH" ALL GUNS CHALLENGE (BO1) - BLACK OPS 3 ZOMBIE CHRONICLES PREP!
Status of UC2GC7O0nULuhHaeItbNPldw exceeded too many failed attempts. If problems persist with this streamer, open an issue here: https://github.com/WolfgangAxel/RedTwiBot/issues/new
Request response was:
Error!

  Line 321 -> coercing to str: need a bytes-like object, dict found

Retrying in one minute.

Edit: Ahh, just checked, there's an Emoji in the Stream title. Any way to have it ignore those? Problem is that it retries 10 times, delaying the sidebar update by 10 minutes because of the Emoji in the title.

kii-chan-reloaded commented 7 years ago

I'll see what I can do about the emoji thing when I get home (at work right now). Could you copy/paste the stream title for me?

For now, there's an update that at least fixes the last line 321 error.

EthanC commented 7 years ago

Stream Title: KINO DER TOTEN REMASTERED GAMEPLAY! 💀 BO3 ZOMBIES CHRONICLES DLC 5 GAMEPLAY

Thank you. Sorry for all the issues, appreciate the help getting it all sorted out.

EthanC commented 7 years ago

Tried running it with the latest update, can't seem to get the sidebar to update, it's just in a loop of checking streams, no errors.

Loading configuration from file...
Configuration file read. Checking for usable values.
Found sections. Checking for values...
Found values for section R
Found values for section T
Found values for section Y
Found values for section M
Found values for section G
Found values for section TS
Found values for section YS
Configuration seems usable. Using C:\Users\SERV\Desktop\sidebarstreamers\configuration.ini
Bot successfully loaded. Entering main loop.
Swiftor is playing The Surge
SirJammy is streaming Old London Mapping Stream #3
Swiftor is playing The Surge
SirJammy is streaming Old London Mapping Stream #3
Swiftor is playing The Surge
SirJammy is streaming Old London Mapping Stream #3

Roughly a minute between each X is streaming Y. Same configuration.ini as last time. I believe I have yet to get a successful sidebar update including YouTube Streams if that helps at all, so it's not something from the latest commit.

kii-chan-reloaded commented 7 years ago

I was unable to recreate your error with the emoji on my machine. Everything works as expected and doesn't crash when any emoji are passed through. After finding that out, I looked into the actual error a bit more, and it turns out that Windows console doesn't use utf-8 encoding by default. Basically, Python handled it perfectly fine, Windows just threw a fit when Python told it to print emoji to the console. There is a workaround for this. Basically, it looks like you would need to run the command chcp 65001 in the console before launching the bot.

As for the whole ten minute delay thing you mentioned earlier, I think I might have figured out the culprit for that, so get the update to see if that helps.

In regards to your most recent comment, your configuration.ini is set up for a 60 second delay between updates, so a minute between each of those lines is expected. Were either of those streamers supposed to be in the sidebar at that point?

EthanC commented 7 years ago

Cool, I'll do that and let ya know, I don't think there's any streams live right now with emojis in the title that I could test it on.

Will keep you updated on the 10 minute delay issue if still existent.

Correct, sorry I didn't explain enough. It was doing 60 seconds between each seperate stream check, and not updating the sidebar.

EDIT: Update, successfully got a YouTube Stream added to the sidebar!

EthanC commented 7 years ago

Also, I believe I found a way to check which game is being played on YouTube Streams, I will update this if I find out how to access it via the API.

Stream I'm currently looking at: https://youtu.be/8sgOo5F5ao0

If you look in the description it says

Game
Year
Explore in YouTube Gaming

image Which leads to this URL. Maybe check if our stream is live, if so then check the lists for our accepted games.

EthanC commented 7 years ago

Okay, more serious issue now, we can go back to the other two after this one is resolved. Sometime within the last 18 hours the bot hit the daily 1,000,000 queries limit. That can't possibly be right? I have 36 YouTube Streams listed, checking once every 1 minute, that's 51,840 MAX queries per day.

I have no idea how long this was happening for, I opened my Server and saw that it did was doing this very fastly for every single YouTube Streamer as long as the console would allow me to go back.

Here's what the console said:

Error with request for DrLilRobot's stream. Attempts remaining: 9/10
Error with request for DrLilRobot's stream. Attempts remaining: 8/10
Error with request for DrLilRobot's stream. Attempts remaining: 7/10
Error with request for DrLilRobot's stream. Attempts remaining: 6/10
Error with request for DrLilRobot's stream. Attempts remaining: 5/10
Error with request for DrLilRobot's stream. Attempts remaining: 4/10
Error with request for DrLilRobot's stream. Attempts remaining: 3/10
Error with request for DrLilRobot's stream. Attempts remaining: 2/10
Error with request for DrLilRobot's stream. Attempts remaining: 1/10
Error with request for DrLilRobot's stream. Attempts remaining: 0/10
Status of UCghRyxbrX8bolFG--KDJ3Hg exceeded too many failed attempts. If problems persist with this streamer, open an issue here: https://github.com/WolfgangAxel/RedTwiBot/issues/new
Request response was:
{'error': {'code': 403, 'message': 'Daily Limit Exceeded. The quota will be reset at midnight Pacific Time (PT). You may monitor your quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/youtube/quotas?project=104144359421', 'errors': [{'extendedHelp': 'https://console.developers.google.com/apis/api/youtube/quotas?project=104144359421', 'domain': 'usageLimits', 'reason': 'dailyLimitExceeded', 'message': 'Daily Limit Exceeded. The quota will be reset at midnight Pacific Time (PT). You may monitor your quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/youtube/quotas?project=104144359421'}]}}
Skipping.
kii-chan-reloaded commented 7 years ago

It looks like my internet is down for some reason right now, so I'm not able to work on this at the moment. I did notice a situation in one of my revisions where if it failed to parse the json properly, it would repeatedly spam Google as fast as it could, but that got fixed in the most recent revision. I don't know if maybe that happened for a while, which made you hit the limit? That's purely speculation. Check that console.google link to see what the load was like. If there's a big spike a day or two ago followed by a lower section, then that's likely what happened. If not, then let me know what your chart looks like. I'm reading through my code, and I'm not coming up with a situation where that is possible on the version I pushed last night.

You might also want to make sure you don't have two bot processes running. I don't know how you've got this set up, but if they run in the background at all, maybe one instance of it is still running on accident?

By my calculations, with 36 streamers, the most you could do is 172800 requests in one day without taking into effect the network delay. That's assuming all 36 streamers return 10 errors a piece at 2 requests per second, which is what it's set up for right now. How it could have hit a million is beyond me right now.

kii-chan-reloaded commented 7 years ago

I ran the bot on my end with your list of youtube streamers, and I got through all 36 with one request each. My Google API manager says I'm hovering at about 0.16 requests/second on average. The sidebar on my testing subreddit updated fine with two YouTube streams, one with emoji:

Streaming now:
    TheRelaxingEnd - My First Kino Camping strategy! BO3 Zombies Chronicles Call of Duty Black Ops 3 DLC5 Gameplay
    Avxry - ❄️'Origins ZOMBIE CHRONICLES' SOLO EASTER EGG & PaP Challenge❄️ (Call of Duty Black Ops 3 Zombies)

As far as I can tell, everything is working great.

I'm looking into the game title thing, and it looks like it's not available through the api. However, I'm working on setting up an additional request that can get the video tags as well. It's still not a perfect solution, but I think looking through both the tags and the video title will help the bot find more streams.

kii-chan-reloaded commented 7 years ago

I should've held off on making that comment. Finished the tag searching and the output from the bot went from 2/3 streams detected to all 3:

Streaming now:
    TheRelaxingEnd - My First Kino Camping strategy! BO3 Zombies Chronicles Call of Duty Black Ops 3 DLC5 Gameplay
    NGTZombies - HYPE* ZOMBIE CHRONICLES ALL DAY MARATHON HYPE
    Avxry - ❄️'Origins ZOMBIE CHRONICLES' SOLO EASTER EGG & PaP Challenge❄️ (Call of Duty Black Ops 3 Zombies)

Made a new update. Everything looks good on my end. We'll see how it holds up on Windows.

EthanC commented 7 years ago

Went ahead and got the latest commit. Generated a New Google API Key just to be safe, going to let it run overnight and see how it goes, hopefully all is well. YouTube and Twitch Streams working great!

Searching tags was a good idea, thanks for the fast implementation.

I'll keep ya updated and let ya know in about 24 hours, then I can hopefully get out of your hair haha. Thanks for all the work.

EthanC commented 7 years ago

Sometime within the last 4 hours I've run into the same issue. Entirely new API Key, so there can't be any other applications using it, two instances running each with their own unique API Key, each having the same issue.

Though, it is worth noting that my second instance is getting this seemingly unique response/error?

GlitchingQueen is streaming DLC 5 Moon 4 Player Easter Egg
Acceptable game not found in title. Making additional request to search tags
Status of UC1j9Sw58UNh-f-21Z-fd8pw exceeded too many failed attempts. If problems persist with this streamer, open an issue here: https://github.com/WolfgangAxel/RedTwiBot/issues/new
Request response was:
{'regionCode': 'US', 'items': [{'id': {'kind': 'youtube#video', 'videoId': 'qIQ1QnH3C24'}, 'kind': 'youtube#searchResult', 'etag': '"m2yskBQFythfE4irbTIeOgYYfBU/dS-yhhjxX7zvP9JkRf2d3F2A2o0"', 'snippet': {'publishedAt': '2017-05-17T03:20:49.000Z', 'channelId': 'UC1j9Sw58UNh-f-21Z-fd8pw', 'description': 'Playing maps on DLC 5 Zombies Chronicles Donate to be on stream https://tinyurl.com/kqart2t.', 'channelTitle': 'Glitching Queen', 'thumbnails': {'medium': {'url': 'https://i.ytimg.com/vi/qIQ1QnH3C24/mqdefault_live.jpg', 'width': 320, 'height': 180}, 'default': {'url': 'https://i.ytimg.com/vi/qIQ1QnH3C24/default_live.jpg', 'width': 120, 'height': 90}, 'high': {'url': 'https://i.ytimg.com/vi/qIQ1QnH3C24/hqdefault_live.jpg', 'width': 480, 'height': 360}}, 'title': 'DLC 5 Moon 4 Player Easter Egg', 'liveBroadcastContent': 'live'}}], 'kind': 'youtube#searchListResponse', 'etag': '"m2yskBQFythfE4irbTIeOgYYfBU/G74ESwusNmGla6hNk4szTE4FCJ4"', 'pageInfo': {'resultsPerPage': 5, 'totalResults': 1}}
Skipping.
Error with request for MrRoflWaffles's stream. Attempts remaining: 9/10

Would it be worth implementing some sort of verbose log to a file so you can see exactly what's going on and at what times? I don't understand what the huge difference between Windows and Linux would be that's causing this issue. Having ports open wouldn't cause the issue right? My server is located in the lab at Malwarebytes Anti-Malware, so obviously all ports are closed, hasn't been an issue yet with any other Bot (even using YouTube/Google API) yet.

Going back as far as I can, it's the same Request Limit error.

Error with request for Syndicate's stream. Attempts remaining: 0/10
Status of UCRbCW5jA-FUfyrauKuu9xuw exceeded too many failed attempts. If problems persist with this streamer, open an issue here: https://github.com/WolfgangAxel/RedTwiBot/issues/new
Request response was:
{'error': {'message': 'Daily Limit Exceeded. The quota will be reset at midnight Pacific Time (PT). You may monitor your quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/youtube/quotas?project=407008487082', 'code': 403, 'errors': [{'extendedHelp': 'https://console.developers.google.com/apis/api/youtube/quotas?project=407008487082', 'domain': 'usageLimits', 'message': 'Daily Limit Exceeded. The quota will be reset at midnight Pacific Time (PT). You may monitor your quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/youtube/quotas?project=407008487082', 'reason': 'dailyLimitExceeded'}]}}

Here's what the console.google.com link looks like. I may be looking at these wrong, but it seems I haven't even came close to the 1M Query limit? Makes me think something is causing it to spam retries on streams, and it hits the 300,000 per 100s limit. image image

Any debugging ideas you have feel free to suggest. One idea I had was setting the retry limit to 1, as it seems something is causing it to continuously retry and that may be hitting the limit super fast, I just can't go back far enough in the console window to see where it starts, which is what a log file would help with.

Really hate how much work this is for you, hopefully we can get to the bottom of it.

kii-chan-reloaded commented 7 years ago

I've updated with a few things:

On the note of the limit hitting, none of what you've showed me seems like it should be causing you to hit the limit. If you've only got 30,000 requests in a 6 hour period, that only comes out to 120,000 a day, and that's well below even the 300,000 / 100sec / user limit. Furthermore, the bot immediately sleeps for 1/2 a second after each request it makes, so it should be impossible for it to break more than 2/second. The flat line from 7ish to 8PM is more or less what I would expect from the bot. I've changed the bot so instead of pausing for 0.5 seconds after each request, it pauses 0.5 seconds before each request, meaning that the spam literally could not possibly happen.

For the stream title thing, the Twitch API reference doesn't have a stream title in the response. I could change the youtube links to use whatever game title they matched on, however that potentially wouldn't format the game name properly due to the workarounds for avoiding : and =.

Also, apparently Twitch released v5 and depreciated v3 of their API less than one month after I made this bot. Looks like I'll need to update that if I want this thing to work past February 2018. Worst part is it now needs ID numbers instead of just account names, so that means I'll have to rework basically everything for Twitch and break compatibility with existing configuration.inis.

I'm not going to be able to work an this further tonight. Maybe in the morning I'll be able to focus more and figure out what's going on.

edit: oh yeah, and I'm going to run my bot with your list of streamers from above overnight and see how my requests/limit hitting compare.

EthanC commented 7 years ago

Nice work, running the latest version now, all is well as of right now because the daily reset was an hour ago, will update in a couple hours before I go to bed.

If the Rate Limit is hit before I go to bed, I'm going to stop the script for the night, and start again in the morning (same day, before the reset), to see if it truly thinks it's hitting the 1M per day Limit, or the message is just misleading and it's somehow hitting some other sort of Rate Limit.

No worries on the Twitch Stream titling, not worth the troubles. I'd be glad to help out in testing with the v5 API, got more than enough time and an entirely different system than you, least I could do after all this trouble I've had ya go through haha.

Do tell me how yours goes, very odd if it goes well and mine does not, can't see any reason why it'd be an issue based on OS.

kii-chan-reloaded commented 7 years ago

I've figured out what's going on, and things don't look too great for youtube support.

The API doesn't just work on just number of requests, there's also something called "quota points" associated with each request. As it stands, each request is costing 100 points, so if the bot were to run without a limit on your current configuration, it would rack up something like 5 million points in one day. The free tier goes up to 1 million a day, By my quick and dirty calculations, to go all day without hitting the limit, the maximum number of channels to search is 6.

The 100 for that one request is ridiculously expensive though, as most other API calls only cost like 3-5 a piece. That being said though, there's not really a replacement for finding live streams. If there is a way to do it, it'll have to be by requesting each channel's activities multiple times and parsing from there. I've had a few ideas about it, but none of what gets returned from the API contains live streams, only the regular uploads.

EthanC commented 7 years ago

Damn, that really sucks to hear. I appreciate all the work on getting it to what it is right now, I guess I'll cut it down to 6 of the most active streamers and leave it at that, better than nothing 😄

Feel free to reach out if you have any other ideas, I'd be glad to test out.

I'm going to run it tomorrow (when my Quota resets) with 6 YouTube Streamers, if no errors we can close this Issue. Only error I'm getting right now is Twitch-related, and I've opened a new Issue for it.

kii-chan-reloaded commented 7 years ago

Well here's some good news- I've almost got the script transitioned over to V5 of the Twitch API, and it will probably come with great improvements, too. I'm working on getting most, if not all, of the streamer's streams in a single request versus one request per streamer. It'll cut the time per loop significantly at your level of streamers (probably something like 2-3 seconds for all Twitch statuses versus the 18-25 it probably takes now). I've also thrown together a script that will take care of converting the current configuration.ini for the V5 change, but I won't put that up until the whole transition is complete. As it stands right now, I'm pretty sure it works, but I can't be certain until some streams are online.

kii-chan-reloaded commented 7 years ago

I've released the update with the new Twitch API. My recommendation is that you make a separate folder for this bot until you know things will work out. Everything seems okay when I run it, but I'm sure I haven't seen all the errors yet. Copy your current configuration into the new folder, then run the upgrader script. The new configuration file it makes won't have the right name- this is just a preventative measure against overwriting a config and losing important information. Make sure that all of your streamers made it into the new config, then delete the copied one and rename configuration.V5.ini to configuration.ini.

Also, I mostly ditched the YouTube API and settled on just scraping each channel's "Live streams" html page. This is pretty expensive in terms of bandwidth (1000+ lines of data to download versus like 15), but it works and there shouldn't be an arbitrary limit on it. The API is still used to look through the stream's tags if the title doesn't contain a game name, though, so don't delete your key. The request it makes for the tag information only costs 3, so by my math, since the bot waits 0.5 seconds between requests it could only hit about 250,000 quota points in one day regardless of how many streamers are in your config, and that's assuming every single YouTuber is streaming for a full 24 hours. Basically, the limit is no longer an issue.

To get YouTube to work, you'll need to install bs4 through pip. This overhaul of YouTube is another reason why I'm suggesting you keep your mostly-functional bot in a different folder. If things turn out differently like they have been, then you have something to fall back on.

EthanC commented 7 years ago

Very nice work, I've gone ahead and updated, successfully got my configuration.ini converted over to v5, everything is up and running as intended.

I've found 1 issue so far. When no acceptable Title or Tags are found for a YouTube Stream, it'll restart it's scan of all Streams, so it gets into a loop of checking streams, never updating the sidebar.

Configuration seems usable. Using C:\Users\SERV\Desktop\SidebarStreamers\configuration.ini
Bot successfully loaded. Entering main loop.
summit1g is playing Borderlands 2
admiralbahroo is playing PLAYERUNKNOWN'S BATTLEGROUNDS
swiftor is playing Call of Duty: Black Ops III
MrDalekJD is streaming KINO DER TOTEN ROUND 100 STARTING ROOM ATTEMPT! - BLACK OPS 3 ZOMBI... (/watch?v=3rOvlFR-KcM)
mcsportzhawk is streaming ORIGINS EASTER EGG NO GUM CHALLENGE 🤖 WITH SUBS! | ZOMBIES CHRONICL... (/watch?v=LjcDsxF0Z7k)
TheSyndicateProject is streaming Black Ops 3: "ORIGINS" - ZOMBIE CHRONICLES *LIVESTREAM* w/ Syndicate! (/watch?v=80nTISu-mKU)
GameSpot is streaming Nex Machina: Death Machine Exclusive Livestream (/watch?v=Suj0nyo1FM8)
Acceptable game not found in title. Making additional request to search tags
summit1g is playing Borderlands 2
admiralbahroo is playing PLAYERUNKNOWN'S BATTLEGROUNDS
swiftor is playing Call of Duty: Black Ops III
MrDalekJD is streaming KINO DER TOTEN ROUND 100 STARTING ROOM ATTEMPT! - BLACK OPS 3 ZOMBI... (/watch?v=3rOvlFR-KcM)
mcsportzhawk is streaming ORIGINS EASTER EGG NO GUM CHALLENGE 🤖 WITH SUBS! | ZOMBIES CHRONICL... (/watch?v=LjcDsxF0Z7k)
TheSyndicateProject is streaming Black Ops 3: "ORIGINS" - ZOMBIE CHRONICLES *LIVESTREAM* w/ Syndicate! (/watch?v=80nTISu-mKU)
GameSpot is streaming Nex Machina: Death Machine Exclusive Livestream (/watch?v=Suj0nyo1FM8)
Acceptable game not found in title. Making additional request to search tags
summit1g is playing Borderlands 2
admiralbahroo is playing PLAYERUNKNOWN'S BATTLEGROUNDS
swiftor is playing Call of Duty: Black Ops III
MrDalekJD is streaming KINO DER TOTEN ROUND 100 STARTING ROOM ATTEMPT! - BLACK OPS 3 ZOMBI... (/watch?v=3rOvlFR-KcM)
mcsportzhawk is streaming ORIGINS EASTER EGG NO GUM CHALLENGE 🤖 WITH SUBS! | ZOMBIES CHRONICL... (/watch?v=LjcDsxF0Z7k)
TheSyndicateProject is streaming Black Ops 3: "ORIGINS" - ZOMBIE CHRONICLES *LIVESTREAM* w/ Syndicate! (/watch?v=80nTISu-mKU)

You can see here that the Gamespot stream is obviously not an accepted Title and doesn't have any acceptable Tags, however the bot continues to restart its search of Streams.

Also, would it be possible to remove the (/watch?v=streamidhere) in the console? Just to make things a little cleaner.

Other than that, all seems well, I will update you soon!

kii-chan-reloaded commented 7 years ago

That doesn't seem to be happening for me, and I'm not seeing anything in the code that would suggest that could happen. How many YouTubers are in your config, and how long of a delay is there between it restarting the search? I made a few slight changes, so try the new version. It will now print out All streamer statuses retrieved. Checking if sidebar should update... before it checks Reddit and ends its loop, so let me know if you don't see that.

EthanC commented 7 years ago

I believe the issue was because I had the time set to 60s and that wasn't enough for it to fetch all streams and attempt to update the sidebar, as putting it to 120s fixed that issue. It ran overnight and all seems well!

EthanC commented 7 years ago

3 days no issues! I think it's safe to close this Issue after this minor error is fixed.

This isn't any problem with the bots connection, I was just playing around with Google's API Settings, got the Bot to throw an error and realized an issue in the Console output.

Error with request for NoahJ456's stream. Attempts remaining: 9/3
Error with request for NoahJ456's stream. Attempts remaining: 8/3
Error with request for NoahJ456's stream. Attempts remaining: 7/3
Status of UCP9tAErY_RlX4RFKssE4ogg exceeded too many failed attempts. If problems persist with this streamer, open an issue here: https://github.com/WolfgangAxel/RedTwiBot/issues/new
Request response was:
{'error': {'code': 403, 'message': 'The calling IP address <redacted> does not match the IP restrictions configured on the API key. Please use the API Console to update your key restrictions.', 'errors': [{'extendedHelp': 'https://console.developers.google.com/apis/credentials?project=<redacted>', 'domain': 'usageLimits', 'message': 'The calling IP address <redacted> does not match the IP restrictions configured on the API key. Please use the API Console to update your key restrictions.', 'reason': 'ipRefererBlocked'}]}}
Error details:
 'items'
Skipping.

Issue here is that the Attempts remaining: counter is off now that it's been lowered to 3.

Again, the issue isn't that it was failing, I did that myself, just the counter :)