Feature request: Ignore Google auth

[Dwarih]

Hey, For the past few weeks I've been randomly getting the same error message mentioned in https://github.com/MarcelRobitaille/bbyen/issues/7. I believe I have too many channels (around 300) in the whitelist which causes it to throw the error message after couple of days. (I've added multiple channels in the last couple of weeks) Bbyen works fine when I reauthenticate but doing that every couple of days is bit bothering.

As I only use the whitelisted channels, would it be possible to add a config option to ignore google authentication when checking for new videos? I'm adding the channels manually so I don't really need to authenticate to Youtube to get my subscriptions.

[MarcelRobitaille]

That's a nice idea. It doesn't really align with the original intent of this project. If you are whitelisting all the channels you want notifications for, you could just subscribe to those RSS feeds. That said, you bring up a good point that I already added a whitelist, and it makes no sense to use the YouTube API when the whitelist is in use.

Could you change your config so we can be absolutely sure you're getting invalid grant when processing the subscriptions? Could you also tell me what format of ID/URL you are using for the whitelist? Some formats need the API just to extract the ID.

    "timers": {
        "subscriptions": "1 days",
        "videos": "1 month",
    },
    "kickoff": {
        "subscriptions": true,
        "videos": false
    },

[Dwarih]

I fully understand that it was not intent of this project. However, as I read in https://github.com/MarcelRobitaille/bbyen/issues/6, it is currently not possible to get emails for only the "All" subscriptions. This is why I went with the whitelisted channels as a workaround which works very well. I used RSS before I found bbyen and I couldn't find anything that would be good replacement for my emails. Bbyen does exactly what I want.

I've just received the invalid_grant message, so I reauthenticated again and set the videos to false in the config. The error shows up usually within 1-2 weeks. We'll see. What exactly does the false do? I received email for the new videos anyway. I expected that the videos would not be checked with that set to false.

My timers are currently set to update more quickly. I understand that 20 minutes might be a bit overkill but I like to be notified quickly haha.

Should I change it to 1 month?

"timers": {
                "subscriptions": "2 days",
                "videos": "20 minutes"

I extract the Channel ID manually so I don't really use the links even though I know they should work. Here's a small snippet of the channels I use.

"whitelistedChannelIds": [
                "UCKpbi30FGG0sWHr0BpRxyaQ",
                "UCrZ3Q0bO1FilIYkn8ahDxSQ",
                "UCKKiQ8Bz3pgiJ7JdqNrQfeQ",
                "UCWoEpiHaC7LOQhaHFT8Rx7A"
        ]
}

[MarcelRobitaille]

Hi @Dwarih.

Regarding the second half of your message: I am trying to check if it's really checking for new subscriptions that's causing your problem. The config changes I requested are only to see if you get the invalid_grant message when only checking for new subscriptions. You can change it back so it updates more frequently after we verify that :slightly_smiling_face:

What exactly does the false do? I received email for the new videos anyway. I expected that the videos would not be checked with that set to false.

The kickoff part of the config only determines if the subscriptions / videos are checked immediately after the program starts or if the program should wait for the specified delay. Since we are trying to stop videos from being checked for now, please set kickoff.videos: false.

My timers are currently set to update more quickly. I understand that 20 minutes might be a bit overkill but I like to be notified quickly haha.

Should I change it to 1 month?

Yes please. I want to see if you will get invalid_grant only when checking for subscriptions. You can make it faster later. You can also do faster than 1 day for subscriptions if you wish.

Please run it for a little while with these settings and let me know if you get invalid_grant within the normal time. You don't have to wait the full month, just around how long it usually takes for you to get the error.

[Dwarih]

Understood, thanks for explanation!

I changed the videos timer as you asked me to do but whenever I do that, bbyen starts freaking out. It works fine whenever I keep my default 20 minutes but as soon as I change it to 1 month, these errors below show up.

Feb 14 22:48:15 Linux systemd[1]: Started YouTube email notification system from Github - bbyen.
Feb 14 22:48:16 Linux systemd-journald[14779]: Suppressed 9234 messages from bbyen.service
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.790Z [info] [subscriptions]: Checking subscriptions... {}
Feb 14 22:48:16 Linux node[4012]: (node:4012) TimeoutOverflowWarning: 2629800000 does not fit into a 32-bit signed integer.
Feb 14 22:48:16 Linux node[4012]: Timeout duration was set to 1.
Feb 14 22:48:16 Linux node[4012]: (Use `node --trace-warnings ...` to show where the warning was created)
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.799Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.800Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.831Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.832Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.836Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.841Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.842Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.846Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.851Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.855Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.865Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.873Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.880Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.914Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.932Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:16 Linux node[4012]: 2023-02-14T21:48:16.978Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.022Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.056Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.066Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.084Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.098Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.107Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.133Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.140Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.160Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.171Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.192Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.273Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.310Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.337Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.351Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.367Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.379Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.391Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.416Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.439Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.471Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.555Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.600Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.637Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.665Z [error] [videos]: Status code 429 {}
Feb 14 22:48:17 Linux node[4012]: Error: Status code 429
Feb 14 22:48:17 Linux node[4012]:     at ClientRequest.<anonymous> (/opt/bbyen/node_modules/rss-parser/lib/parser.js:88:25)
Feb 14 22:48:17 Linux node[4012]:     at Object.onceWrapper (node:events:642:26)
Feb 14 22:48:17 Linux node[4012]:     at ClientRequest.emit (node:events:527:28)
Feb 14 22:48:17 Linux node[4012]:     at HTTPParser.parserOnIncomingClient (node:_http_client:631:27)
Feb 14 22:48:17 Linux node[4012]:     at HTTPParser.parserOnHeadersComplete (node:_http_common:128:17)
Feb 14 22:48:17 Linux node[4012]:     at TLSSocket.socketOnData (node:_http_client:494:22)
Feb 14 22:48:17 Linux node[4012]:     at TLSSocket.emit (node:events:527:28)
Feb 14 22:48:17 Linux node[4012]:     at addChunk (node:internal/streams/readable:315:12)
Feb 14 22:48:17 Linux node[4012]:     at readableAddChunk (node:internal/streams/readable:289:9)
Feb 14 22:48:17 Linux node[4012]:     at TLSSocket.Readable.push (node:internal/streams/readable:228:10)
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.671Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.710Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.721Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.744Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.768Z [info] [videos]: Checking for new videos... {}
Feb 14 22:48:17 Linux node[4012]: 2023-02-14T21:48:17.785Z [info] [videos]: Checking for new videos... {}

[MarcelRobitaille]

Welp, TIL that setInterval only accepts a 32-bit integer. 24 days seems to be about the limit. Could you try with like 1 week for timers.videos? Sorry about that.

[Dwarih]

Okay, setting 1 week works. Will report back once or if the invalid_grant happens.

[MarcelRobitaille]

Thank you for doing this. I know it's annoying to not have the videos running. If you're willing to set it up, you could run 2 instances with 2 different credentials: one to test this and one running the videos like normal.

[Dwarih]

It's fine. I managed without it. So I used these settings and the invalid_grant error happened on 22nd of February. It basically took the entire week. Any guess what might be causing it?

        "timers": {
                "subscriptions": "2 days",
                "videos": "1 week"
        },
        "kickoff": {
                "subscriptions": true,
                "videos": false
        },

Feb 21 07:04:50 Linux node[13350]: 2023-02-21T06:04:50.247Z [warn] [subscriptions]: Ignoring channel not in whitelacklist: 조꽁드Joconde's baking (UC6cH7oaIvWYa_0iATEEynDQ) {}
Feb 21 07:04:50 Linux node[13350]: 2023-02-21T06:04:50.248Z [info] [subscriptions]: Done checking subscriptions... {}
Feb 22 07:04:46 Linux node[13350]: 2023-02-22T06:04:46.527Z [info] [videos]: Checking for new videos... {}
Feb 22 07:04:47 Linux node[13350]: 2023-02-22T06:04:47.110Z [error] [videos]: invalid_grant {"response":{"config":{"method":"POST","url":"https://oauth2.googleapis.com/token","data":"refresh_token=1%2F%2F09h4hvch4m3UqCgYIARAAGAkSNwF-L9
Feb 22 07:04:47 Linux node[13350]: Error: invalid_grant
Feb 22 07:04:47 Linux node[13350]:     at Gaxios._request (/opt/bbyen/node_modules/gaxios/build/src/gaxios.js:130:23)
Feb 22 07:04:47 Linux node[13350]:     at runMicrotasks (<anonymous>)
Feb 22 07:04:47 Linux node[13350]:     at processTicksAndRejections (node:internal/process/task_queues:96:5)
Feb 22 07:04:47 Linux node[13350]:     at async OAuth2Client.refreshTokenNoCache (/opt/bbyen/node_modules/google-auth-library/build/src/auth/oauth2client.js:178:19)
Feb 22 07:04:47 Linux node[13350]:     at async OAuth2Client.getRequestMetadataAsync (/opt/bbyen/node_modules/google-auth-library/build/src/auth/oauth2client.js:298:17)
Feb 22 07:04:47 Linux node[13350]:     at async OAuth2Client.requestAsync (/opt/bbyen/node_modules/google-auth-library/build/src/auth/oauth2client.js:371:23)
Feb 22 07:04:47 Linux node[13350]:     at async Timeout.parseFeedsAndNotify [as _onTimeout] (/opt/bbyen/src/videos.js:61:23)

Also, I just noticed that there's a typo whitelacklist instead of whitelist in the logs

[MarcelRobitaille]

This tells me it's checking for videos that is giving you the invalid grant error. What you propose in the issue of not use the Google API to get your subscriptions would not work. In your case, it's using the YouTube API to check for new videos from those subscriptions.

Some things you could try are:

• Check in the Google console to see if you're hitting any quota limit.
• Make a new Google account and create your API keys in that. If you're using whitelisted channels anyway, you won't need to transfer subscriptions or anything. I've hit bugs with the YouTube API before and their official response was since the API works if I create a new account and it's only a problem with my account they won't fix it.

It would be possible to scrape the page's HTML instead of using the YouTube API, but that would require a significant rewrite of the project. I am considering it because the YouTube API is a pain to deal with.

[MarcelRobitaille]

I will close this due to inactivity. @Dwarih Please reopen if you are still experiencing this.